Skip to main content

Web service

Header image

 

Web services specifications

This page describes the specifications for web service consultation of farm, survey and crop data from the RTB project.

Security, nature and format of exchanges

All exchanges must use TLS. The exchanged data are serialized in JSON.

Exchanges are done according to the JSON:API specification, see the documentation on drupal.org. The request must use the following headers (GET requests do not seem to require them):

Accept: application/vnd.api+json
Content-Type: application/vnd.api+json

The JSON response contains three main sections: jsonapi (metadata of no great importance in our case), data (list of entities corresponding to the request made) and links (useful in particular for pagination).

Each content item returned can include the following keys:

  • type: composed of the entity type and the subtype, separated by two dashes, for example survey--survey or crop--banana.
  • id: UUID of the content (sequence of alphanumeric characters). It is this identifier and not the numeric identifier that is used in the JSON:API.
  • links: list of links, including a link to retrieve the individual element (links.self.href).
  • attributes: values of fields which are not references.
  • relationships: references to other entities: users, taxonomy terms, etc.

At present, only read requests (GET) are accepted. However, it is possible to accept requests for creation (POST), modification (PATCH) or deletion (DELETE), by modifying the corresponding parameter in the interface (/admin/config/services/jsonapi).

JSON:API only responds favourably to a request if the rights of the user who initiated it are sufficient. For example, one can only retrieve one or more entities if one has the right to see published entities of the corresponding type.

The JSON:API Extras module, installed on this site, allows you to disable certain undesirable resources in the context of this project, such as the list of users of the site (see /admin/config/services/jsonapi/resource_types).

Entity types

The data we are interested in are of three types: farm, survey and crop. The table below lists the machine names as well as those of the subtypes. Only crops can be broken down into separate subtypes (bundles).

Name Machine name Sub-types
Farm farm farm
Survey survey survey
Crop crop cassava, banana, potato, yam

Farms

The following fields are attached to a farm:

Name Type Base field ? Description
user_id Integer  yes Identifier of the author of this content
name String yes Name of the holding or farm
status Boolean yes Is the entity published?
created Timestamp Unix yes Date de created
changed Timestamp Unix yes Date modified

Only users with the administrator role can see the name of a farm (which is very often the name of an individual in the context of this project), by any means (consultation via web service or via the interface in particular). The access control is done in source:docroot/profiles/rtb/modules/custom/rtb_survey/rtb_survey.module@7b80ea3f#L239 (function rtb_survey_entity_field_access()).

For reasons mainly related to the previous states of the project (protection of linked personal data, now solved by the above access control), a farm is not accessible (via the interface or web service) by an anonymous user and requires authentication (administrator or oauth role).

 

Surveys

A survey has the following fields attached to it:

Name Type Base field ? Description
user_id Reference to an entity yes Identifier of the author of this content
name String yes Name of the survey
status Boolean yes Is the entity published?
created Timestamp Unix yes Date created
changed Timestamp Unix yes Date modified
survey_date Timestamp Unix yes Date of the survey
crop_type Reference to an entity yes Identifier of the taxonomy term belonging to the crop_type vocabulary
farm Reference to an entity yes Identifier of the farm linked to this survey
field_country Country (module Country) no Country where the survey was conducted
field_province String no Province
field_district String no Region
field_location_name String no Location name
field_location Geofield no Geographical coordinates
field_minimum_altitude Integer no Minimum altitude
field_maximum_altitude Integer no Maximum altitude
field_land_use Reference to an entity no Identifier of the taxonomy term belonging to the land_use vocabulary
field_habitat_type Reference to an entity no Identifier of the taxonomy term belonging to the habitat_type vocabulary

The surveys are accessible in consultation (via the interface or web service) to all and do not require authentication. If in the future we wish to allow the creation, modification or deletion of a survey by web service, we will have to adjust the rights accordingly (in this case authentication will of course be necessary).

Crops

The following fields are attached to a crop:

Name Type Base field ? Description
user_id Reference to an entity yes Identifier of the author of this content
name String yes Name of this crop
status Boolean yes Is the entity published?
created Timestamp Unix yes Date created
changed Timestamp Unix yes Date modified
meaning String yes Meaning of the vernacular name
putative_classification String yes Assumed classification (usually in the field, before confirmation)
taxonomy Reference to an entity yes Taxon, reference to a term in the taxonomy vocabulary
survey Reference to an entity yes Identifier of the survey related to this crop
picture Image yes Image transferred
reviewed Boolean yes True if the observation has been validated
field_cultivation_cycle String no Crop cycle (bananas only)
field_misc Long text no Miscellaneous observations
field_iucn_national Reference to an entity no IUCN classification (iucn vocabulary)
field_iucn_national String no Explanation of the origin of the variety
field_seeds_in_fruits String no Presence of seeds in the fruit (bananas only)
field_synonyms Reference to a paragraph no Synonyms (common names)
field_use Reference to an entity no Use of fruit, vocabulary use
field_use_other String no Use of other parts of the plant (bananas only)
field_why_like String no Reason for liking the variety

The crops are accessible for consultation (via the interface or web service) to all and do not require authentication. If in the future we wish to allow the creation, modification or deletion of a crop by web service, we will have to adjust the rights accordingly (in this case authentication will of course be necessary).

Authentification

The Simple OAuth module installed on this site allows users to be authenticated using the OAuth 2.0 protocol. For machine-to-machine use, the client credentials grant mode is suitable. The URL for obtaining the token is /oauth/token.

Configuration

  1. Go to the certificates directory at the root of the project (create it if necessary) and run the following two commands to generate a public and private key pair:
    openssl genrsa -out private.key 2048
    openssl rsa -in private.key -pubout > public.key
  2. In admin/config/people/simple_oauth, enter the values ../certificates/public.key and ../certificates/private.key for "public key" and "private key" respectively. Warnings about the rights to these files may appear on the screen. Make them readable only for the owner and the web server.

Creating access

Authentication via OAuth is based on the primary mechanism for managing user authentication. Create a user and assign only the "OAuth" role in addition to the "authenticated user" role. Then, at admin/config/services/consumer, create a new consumer or modify the default one. Attach it to the previously created user using the autocomplete mechanism, enter a secret (which must be remembered, it will be the secret key to be transmitted to the user), check Is confidential, Is this consumer 3rd party, as well as OAuth under Scopes, then save. The client key is the UUID displayed in the list of consumers

Retrieving a token and making signed requests

Manually

Make a POST request to /oauth/token with the values for grant_type, client_id and client_secret as parameters (in the request body). Example with curl :

curl -X POST -d "grant_type=client_credentials&client_id=CLIENT_ID&client_secret=CLIENT_SECRET" https://rtb.crop-diversity.org/oauth/token

Replacing CLIENT_ID with the base 64 string (e.g. bcf058d6-932f-4745-a4a5-66c032a62de5) and CLIENT_SECRET with the password provided. If the request is successful, it results in a JSON response of the following type:

{
  "token_type":"Bearer",
  "expires_in":300,
  "access_token":"LONG_BASE64_STRING" 
}

The access token then allows authentication of the requests by adding the following header:

Authorization: Bearer LONG_BASE64_STRING

In Drupal 7

The HTTP client module allows to make signed requests if it is accompanied by the OAuth module (not tested with OAuth 2.0).

Under Drupal ≥ 8 with Guzzle

Install kamermans/guzzle-oauth2-subscriber (compose require kamermans/guzzle-oauth2-subscriber). Declare a client and a client factory in the file mymodule.services.yml :
 

services:
  mymodule.http_client:
    class: GuzzleHttp\Client
    factory: mymodule.http_client_factory:get
  mymodule.http_client_factory:
    class: Drupal\mymodule\MyModuleClientFactory
    #arguments: []

Then in src/MyModuleClientFactory.php (examples adapted from existing code, untested, without comments or biclef management, which should not be hard-coded) :

<?php

namespace Drupal\mymodule;

use GuzzleHttp\Client;
use GuzzleHttp\HandlerStack;
use kamermans\OAuth2\GrantType\ClientCredentials;
use kamermans\OAuth2\OAuth2Middleware;

/**
 * HTTP client management to interact whith Questions services.
 */
class MyModuleClientFactory {

  /**
   * Returns a configured client object for Questions web services.
   */
  public function get() {
    $stack = HandlerStack::create();
    // Authorization client: this is used to request OAuth access tokens.
    $reauth_client = new Client([
      'base_uri' => 'https://rtb.crop-diversity.org/oauth/token',
      //'verify' => FALSE,
    ]);
    $reauth_config = [
      'client_id' => 'CLIENT_ID',
      'client_secret' => 'CLIENT_SECRET',
    ];
    $grant_type = new ClientCredentials($reauth_client, $reauth_config);
    $middleware = new OAuth2Middleware($grant_type);
    $stack->push($middleware);

    $client = new MyModuleClient([
      'base_uri' => 'https://rtb.crop-diversity.org/jsonapi/',
      'handler' => $stack,
      'auth' => 'oauth',
      'headers' => [
        'Content-Type' => 'application/vnd.api+json',
        'Accept' => 'application/vnd.api+json',
      ],
    ]);

    return $client;
  }

}

As we can see, as part of JSON:API, I'm overriding the client itself and in particular forcing Guzzle to use the correct headers. Here is the code for src/MyModuleClient.php :

<?php

namespace Drupal\mymodule;

use Drupal\Component\Serialization\Json;
use GuzzleHttp\Client;

/**
 * Make serialization transparent.
 *
 * We cannot use the json key as Guzzle would automatically override the
 * Content-Type header.
 */
class MyModuleClient extends Client {

  /**
   * {@inheritdoc}
   */
  public function request($method, $uri = '', array $options = []) {
    if (isset($options['body'])) {
      $options['body'] = Json::encode($options['body']);
    }

    return parent::request($method, $uri, $options);
  }

}

All that remains is to inject the mymodule.http_client and serialization.json services (for example in the $myClient and $serializer properties of the current class respectively) to make a request :

try {
  $response = $this->myClient->get('survey/survey');
}
catch (RequestException $e) {
  // ...
}
$surveys = (string) $response->getBody();
$surveys = $this->serializer->decode($surveys);

Lookup queries

It is assumed that this will be the most commonly used query type for the time being. Do not hesitate to refer to the official documentation of the Drupal JSON:API module.

Retrieving data from a single entity

Retrieve the set of resources for a given entity and entity sub-type:

GET /jsonapi/survey/survey

(set of surveys)

GET /jsonapi/crop/banana

(set of bananas)

We can see that the path always starts with jsonapi, followed by the machine name of the entity, then that of the sub-entity. Please note that the data received is paginated, with a maximum of 50 elements per page. Refer to the URLs in the links section of the response to iterate if necessary (see documentation).

Retrieve data from a single entity

GET /jsonapi/survey/survey/UUID

Where UUID is the identifier that can be seen for example in the list of all surveys.

Modify this section
Filter
To search for all surveys in Papua New Guinea, for example

GET /jsonapi/survey/survey?filter[field_country]=PG

See documentation on filtering and sorting.

Our partners

International potato center (CIP) logo
Bioversity logo
International center for tropical agriculture (CIAT) logo
French agricultural research centre for international development (CIRAD) logo
International institute of tropical agriculture (IITA) logo