Web services specifications

This page describes the specifications for web service consultation of farm, survey and crop data from the RTB project.

Security, nature and format of exchanges

All exchanges must use TLS. The exchanged data are serialized in JSON.

Exchanges are done according to the JSON:API specification, see the documentation on drupal.org. The request must use the following headers (GET requests do not seem to require them):

Accept: application/vnd.api+json
Content-Type: application/vnd.api+json

The JSON response contains three main sections: jsonapi (metadata of no great importance in our case), data (list of entities corresponding to the request made) and links (useful in particular for pagination).

Each content item returned can include the following keys:

• type: composed of the entity type and the subtype, separated by two dashes, for example survey--survey or crop--banana.
• id: UUID of the content (sequence of alphanumeric characters). It is this identifier and not the numeric identifier that is used in the JSON:API.
• links: list of links, including a link to retrieve the individual element (links.self.href).
• attributes: values of fields which are not references.
• relationships: references to other entities: users, taxonomy terms, etc.

At present, only read requests (GET) are accepted. However, it is possible to accept requests for creation (POST), modification (PATCH) or deletion (DELETE), by modifying the corresponding parameter in the interface (/admin/config/services/jsonapi).

JSON:API only responds favourably to a request if the rights of the user who initiated it are sufficient. For example, one can only retrieve one or more entities if one has the right to see published entities of the corresponding type.

The JSON:API Extras module, installed on this site, allows you to disable certain undesirable resources in the context of this project, such as the list of users of the site (see /admin/config/services/jsonapi/resource_types).

Entity types

The data we are interested in are of three types: farm, survey and crop. The table below lists the machine names as well as those of the subtypes. Only crops can be broken down into separate subtypes (bundles).

Farms

The following fields are attached to a farm:

Only users with the administrator role can see the name of a farm (which is very often the name of an individual in the context of this project), by any means (consultation via web service or via the interface in particular). The access control is done in source:docroot/profiles/rtb/modules/custom/rtb_survey/rtb_survey.module@7b80ea3f#L239 (function rtb_survey_entity_field_access()).

For reasons mainly related to the previous states of the project (protection of linked personal data, now solved by the above access control), a farm is not accessible (via the interface or web service) by an anonymous user and requires authentication (administrator or oauth role).

Surveys

A survey has the following fields attached to it:

The surveys are accessible in consultation (via the interface or web service) to all and do not require authentication. If in the future we wish to allow the creation, modification or deletion of a survey by web service, we will have to adjust the rights accordingly (in this case authentication will of course be necessary).

Crops

The following fields are attached to a crop:

The crops are accessible for consultation (via the interface or web service) to all and do not require authentication. If in the future we wish to allow the creation, modification or deletion of a crop by web service, we will have to adjust the rights accordingly (in this case authentication will of course be necessary).

Authentification

The Simple OAuth module installed on this site allows users to be authenticated using the OAuth 2.0 protocol. For machine-to-machine use, the client credentials grant mode is suitable. The URL for obtaining the token is /oauth/token.

Configuration

1. Go to the certificates directory at the root of the project (create it if necessary) and run the following two commands to generate a public and private key pair:

   openssl genrsa -out private.key 2048
   openssl rsa -in private.key -pubout > public.key

2. In admin/config/people/simple_oauth, enter the values ../certificates/public.key and ../certificates/private.key for "public key" and "private key" respectively. Warnings about the rights to these files may appear on the screen. Make them readable only for the owner and the web server.

Creating access

Authentication via OAuth is based on the primary mechanism for managing user authentication. Create a user and assign only the "OAuth" role in addition to the "authenticated user" role. Then, at admin/config/services/consumer, create a new consumer or modify the default one. Attach it to the previously created user using the autocomplete mechanism, enter a secret (which must be remembered, it will be the secret key to be transmitted to the user), check Is confidential, Is this consumer 3rd party, as well as OAuth under Scopes, then save. The client key is the UUID displayed in the list of consumers

Retrieving a token and making signed requests

Manually

Make a POST request to /oauth/token with the values for grant_type, client_id and client_secret as parameters (in the request body). Example with curl :

curl -X POST -d "grant_type=client_credentials&client_id=CLIENT_ID&client_secret=CLIENT_SECRET" https://rtb.crop-diversity.org/oauth/token

Replacing CLIENT_ID with the base 64 string (e.g. bcf058d6-932f-4745-a4a5-66c032a62de5) and CLIENT_SECRET with the password provided. If the request is successful, it results in a JSON response of the following type:

{
  "token_type":"Bearer",
  "expires_in":300,
  "access_token":"LONG_BASE64_STRING" 
}

The access token then allows authentication of the requests by adding the following header:

Authorization: Bearer LONG_BASE64_STRING

In Drupal 7

The HTTP client module allows to make signed requests if it is accompanied by the OAuth module (not tested with OAuth 2.0).

Under Drupal ≥ 8 with Guzzle

Install kamermans/guzzle-oauth2-subscriber (compose require kamermans/guzzle-oauth2-subscriber). Declare a client and a client factory in the file mymodule.services.yml :
 

services:
  mymodule.http_client:
    class: GuzzleHttp\Client
    factory: mymodule.http_client_factory:get
  mymodule.http_client_factory:
    class: Drupal\mymodule\MyModuleClientFactory
    #arguments: []

Then in src/MyModuleClientFactory.php (examples adapted from existing code, untested, without comments or biclef management, which should not be hard-coded) :

<?php

namespace Drupal\mymodule;

use GuzzleHttp\Client;
use GuzzleHttp\HandlerStack;
use kamermans\OAuth2\GrantType\ClientCredentials;
use kamermans\OAuth2\OAuth2Middleware;

/**
 * HTTP client management to interact whith Questions services.
 */
class MyModuleClientFactory {

  /**
   * Returns a configured client object for Questions web services.
   */
  public function get() {
    $stack = HandlerStack::create();
    // Authorization client: this is used to request OAuth access tokens.
    $reauth_client = new Client([
      'base_uri' => 'https://rtb.crop-diversity.org/oauth/token',
      //'verify' => FALSE,
    ]);
    $reauth_config = [
      'client_id' => 'CLIENT_ID',
      'client_secret' => 'CLIENT_SECRET',
    ];
    $grant_type = new ClientCredentials($reauth_client, $reauth_config);
    $middleware = new OAuth2Middleware($grant_type);
    $stack->push($middleware);

    $client = new MyModuleClient([
      'base_uri' => 'https://rtb.crop-diversity.org/jsonapi/',
      'handler' => $stack,
      'auth' => 'oauth',
      'headers' => [
        'Content-Type' => 'application/vnd.api+json',
        'Accept' => 'application/vnd.api+json',
      ],
    ]);

    return $client;
  }

}

As we can see, as part of JSON:API, I'm overriding the client itself and in particular forcing Guzzle to use the correct headers. Here is the code for src/MyModuleClient.php :

<?php

namespace Drupal\mymodule;

use Drupal\Component\Serialization\Json;
use GuzzleHttp\Client;

/**
 * Make serialization transparent.
 *
 * We cannot use the json key as Guzzle would automatically override the
 * Content-Type header.
 */
class MyModuleClient extends Client {

  /**
   * {@inheritdoc}
   */
  public function request($method, $uri = '', array $options = []) {
    if (isset($options['body'])) {
      $options['body'] = Json::encode($options['body']);
    }

    return parent::request($method, $uri, $options);
  }

}

All that remains is to inject the mymodule.http_client and serialization.json services (for example in the $myClient and $serializer properties of the current class respectively) to make a request :

try {
  $response = $this->myClient->get('survey/survey');
}
catch (RequestException $e) {
  // ...
}
$surveys = (string) $response->getBody();
$surveys = $this->serializer->decode($surveys);

Lookup queries

It is assumed that this will be the most commonly used query type for the time being. Do not hesitate to refer to the official documentation of the Drupal JSON:API module.

Retrieving data from a single entity

Retrieve the set of resources for a given entity and entity sub-type:

GET /jsonapi/survey/survey

(set of surveys)

GET /jsonapi/crop/banana

(set of bananas)

We can see that the path always starts with jsonapi, followed by the machine name of the entity, then that of the sub-entity. Please note that the data received is paginated, with a maximum of 50 elements per page. Refer to the URLs in the links section of the response to iterate if necessary (see documentation).

Retrieve data from a single entity

GET /jsonapi/survey/survey/UUID

Where UUID is the identifier that can be seen for example in the list of all surveys.

Modify this section
Filter
To search for all surveys in Papua New Guinea, for example

GET /jsonapi/survey/survey?filter[field_country]=PG

See documentation on filtering and sorting.