gafs-manager-api

Contents

Component description

The "gafs-manager" provides a JSON-RPC based interface which allows to interact with a GAFS/XtreemFS installation. It is used by other Contrail components within a Contrail deployment e.g., to create and delete volumes for user storage or make a GAFS/XtreemFS installation part of an application VIN.

It is implemented as Java Servlet which should be run on the same machine as the DIR service of the XtreemFS installation. The code is available in the Contrail OW2 SVN repository in the directory /trunk/resource/gafs-manager/.

Since January 2013 the binary package "contrail-gafs-manager" is available from the Contrail repository. Just install this package on the same node as the DIR service runs (package "xtreemfs-server").

Resources used

  • Tomcat6 
  • HTTP port 8080
  • XtreemFS >= 1.3.1 (xtreemfs-server package)

Public API - gafs-manager-api

List of gafs-manager-api resource sections:

  • listVolumes
  • createVolume  volume_name owner owner_groupname mode [policy factor flag]
  • deleteVolume  volume_name 
  • listOSDsAndAttributes
  • getOSDSelectionPolicies volume_name
  • setOSDSelectionPolicies volume_name policies
  • getReplicaSelectionPolicies volume_name
  • setReplicaSelectionPolicies volume_name policies
  • setPolicyAttribute volume_name attribute_name attribute_value
  • listPolicyAttributes volume_name
  • listACLs volume_name
  • setACL volume_name user_name user_accessrights
  • removeACL volume_name user_name
  • getServers volume_name
  • startVINAgents server_ips vin_agent_identifiers ibis_pool_name ibis_server_address

Parameter in brackets like [policy] indicate optional parameters.

Resource URL: /executeMethod

A remote method is invoked by sending a POST request to the tomcat URL
http://localhost:8080/xtreemfs-jsonrpc/executeMethod
The request must contain a JSON-RPC Version 2.0 message, which must contain the following properties:

  • method: A string containing the method to be invoked.
  • params: The objects to be passed to the invoked method.
  • id: A unique id used to match the response with the request.
  • jsonrpc: The version of the used protocol. This should always be 2.0.

Request:

{
 "method":...,
 "params":{...},
 "jsonrpc":"2.0",
 "id":"1"
}

The gafs-manager replies with a response to all received requests. A valid response contains the following properties:

  • result: Returns the data of the invoked method, if no error occurred, otherwise null.
  • error: Returns the error code, if an error occured while invoking the method, otherwise null.
  • id: The corresponding request id it is responding to.
  • jsonrpc: The version of the used protocol. This should always be 2.0.

Response:

{
 "result":...,
 "jsonrpc":"2.0",
 "id":"1"
}

Examples

A simple request response example, which creates a volume named "testVolume":

Request:
{
 "method": "createVolume",
 "params":{"volume_name":"testVolume", "owner":"myUser", "owner_group":"myGroup", "mode":"0777"},
 "jsonrpc":"2.0",
 "id":"1"
}
Response:
{
 "result":"volume testVolume created.",
 "jsonrpc":"2.0",
 "id":"1"
}

A simple request error example, which produces an error:

Request:
{
 "method":"createVolume",
 "params":{"volume_name":""},
 "jsonrpc":"2.0",
 "id":"1"
}
Response:
{
 "error":{"message":"Null value for parameter 'volume_name' not allowed","code":-32700},
 "jsonrpc":"2.0",
 "id":"2"
}

Configure the GAFS-manager for SSL-support

The GAFS-manager has to be configured to connect to the GAFS using SSL.

Edit the GAFS-manager configuration file webapp/WEB-INF/default_dir and set ssl.enabled to true. To allow for the GAFS manager to create or delete volumes on behalf of a user, an admin_password has to be configured in all GAFS services (DIR, MRC, OSD). This password has to be provided to the GAFS manager. And a valid certificate is needed for authentication to the GAFS services.

The configuration should look similar to:

# administrator password for privileged operations
# set this to the same value as in the mrcconfig.properties
admin_password = admin_passphrase

# specify whether SSL is required
ssl.enabled = true

# server credentials for SSL handshakes
ssl.service_creds = pathToCert/cert.p12
ssl.service_creds.pw = passphrase
ssl.service_creds.container = pkcs12

JSON-RPC method: listVolumes

Description

Returns the list of volumes available in GAFS/XtreemFS.

Syntax

listVolumes

Parameters

No parameters required.

Return Value Format

String[] : list of volume names

Example

Request:
{
 "method":"listVolumes",
 "params":[],
 "jsonrpc":"2.0",
 "id":"1"
}
Response:
{
 "result":["testVolume1","testVolume2"],
 "jsonrpc":"2.0",
 "id":"1"
}

JSON-RPC method: createVolume

Description

Create a volume in GAFS/XtreemFS.

Syntax

createVolume volume_name owner owner_groupname mode [policy factor flag]

Parameters

  • volume_name: the name of the volume to be created.
  • owner: the name or UUID of the owner of the volume to be created.
  • owner_groupname: the owning group name.
  • mode: octal mode of the volume to be created (e.g. 0777).
  • policy: replication policy.
  • factor: replication factor.(REQUIRED if policy is chosen)
  • flag: flag for the chosen replication policy.(REQUIRED if policy is chosen)
  • password: Note, that the password has to be configured by the admin_password-parameter within the default_dir-config and can not be passed as a parameter anymore. (OPTIONAL)

Return Value Format

String : status message

Example using X.509 certificates

Assuming user CN=oauth-java-vepclient1,O=INRIA,ST=Some-State,C=FR

Request:
{
 "method":"createVolume",
 "params":{"volume_name":"testVolume", "owner":"oauth-java-vepclient1", "owner_groupname":"oauth-java-vepclient1", "mode":"0777"},
 "jsonrpc":"2.0",
 "id":"1"
}
Response:
{
 "result":"volume policy_volume created.",
 "jsonrpc":"2.0",
 "id":"1"
}

Example

Assuming user myUser in group myGroup

Request:
{
 "method":"createVolume",
 "params":{"volume_name":"testVolume", "owner":"myUser", "owner_groupname":"myGroup", "mode":"0777"},
 "jsonrpc":"2.0",
 "id":"1"
}
Response:
{
 "result":"volume policy_volume created.",
 "jsonrpc":"2.0",
 "id":"1"
}

JSON-RPC method: deleteVolume

Description

Deletes a volume in GAFS/XtreemFS.

Syntax

deleteVolume volume_name [password]

Parameters

  • volume_name: the name of the volume to be deleted.
  • password: Note, that the password has to be configured by the admin_password-parameter within the default_dir-config and can not be passed as a parameter anymore. (OPTIONAL)

Return Value Format

String : status message

Example

Request:
{
 "method":"deleteVolume",
 "params":{"volume_name":"policy_volume"},
 "jsonrpc":"2.0",
 "id":"1"
}
Response:
{
 "result":"volume policy_volume deleted.",
 "jsonrpc":"2.0",
 "id":"1"
}

JSON-RPC method: listOSDsAndAttributes

Description

Lists available OSDs in GAFS/XtreemFS and their custom attributes. Custom attributes start with the prefix "config." and can be used in combination with policies. Customs attributes are used for customizing the file placement e.g., to store data only in particular countries.

An OSD is available if it is registered to the GAFS/XtreemFS DIR and it was reachable within the last 5 minutes.

Syntax

listOSDsAndAttributes

Parameters

No parameters required.

Return Value Format

Map<String,Map<String,String>>: map with OSD uuids and their attributes as map.

Example

Request:
{
 "method":"listOSDsAndAttributes",
 "params":[],
 "jsonrpc":"2.0",
 "id":"5"
}
Response:
{
 "result":{"UUID:localhost:33640":{"config.country":"DE"}},
 "jsonrpc":"2.0",
 "id":"5"
}

JSON-RPC method: getOSDSelectionPolicies

Description

Lists OSD selection policies on that volume.

Syntax

getOSDSelectionPolicies volume_name

Parameters

  • volume_name: the name of the volume to list policies.

Return Value Format

String : OSD selection policies separated by a ','

Example

Request:
{
 "method":"getOSDSelectionPolicies",
 "params":{"volume_name":"policy_volume"},
 "jsonrpc":"2.0",
 "id":"3"
}
Response:
{
 "result":"1000,3002",
 "jsonrpc":"2.0",
 "id":"3"
}

JSON-RPC method: getReplicaSelectionPolicies

Description

Lists replica selection policies on that volume.

Syntax

getReplicaSelectionPolicies volume_name

Parameters

  • volume_name: the name of the volume to list policies.

Return Value Format

String : replica selection policies separated by a ','

Example

Request:
{
 "method":"getReplicaSelectionPolicies",
 "params":{"volume_name":"policy_volume"},
 "jsonrpc":"2.0",
 "id":"4"
}
Response:
{
 "result":"1001,3002",
 "jsonrpc":"2.0",
 "id":"4"
}

JSON-RPC method: setOSDSelectionPolicies

Description

Set OSD selection policies. See the XtreemFS user guide for a list of available policies.

Syntax

setOSDSelectionPolicies volume_name policies

Parameters

  • volume_name: the name of the volume to list policies.
  • policies: the list of policies to be set on the volume.

Return Value Format

String : status message

Example

Request:
{
 "method":"setOSDSelectionPolicies",
 "params":{"volume_name":"policy_volume","policies":"1002,3002"},
 "jsonrpc":"2.0",
 "id":"1"
}
Response:
{
 "result":{
   "message":"OSD selection policy on policy_volume set to 1002,300",
   "usable_osds":3,
   "volume_name":"policy_test_volume",
   "policy":"1002,3002"
  },
 "jsonrpc":"2.0",
 "id":"1"
}

JSON-RPC method: setReplicaSelectionPolicies

Description

Set replica selection policies. See the XtreemFS user guide for a list of available policies.

Syntax

setReplicaSelectionPolicies volume_name policies

Parameters

  • volume_name: the name of the volume to list policies.
  • policies: the list of policies to be set on the volume.

Return Value Format

String : status message

Example

Request:
{
 "method":"setReplicaSelectionPolicies",
 "params":{"volume_name":"policy_volume","policies":"1000,3002"},
 "jsonrpc":"2.0",
 "id":"1"
}
Response:
{
 "result":
  {
   "message":"Replica selection policy on policy_test_volume set to 1000,3002",
   "usable_osds":3,
   "volume_name":"policy_test_volume",
   "policy":"1000,3002"
  }  
 "jsonrpc":"2.0",
 "id":"1"
}

JSON-RPC method: listPolicyAttributes

Description

Lists custom attribute filters on policies like 1000.country=DE.
See the XtreemFS user guide for more examples.

Syntax

listPolicyAttributes volume_name

Parameters

  • volume_name: the name of the volume to list attribute filters.

Return Value Format

String[] : list of policy attributes. each one of the form policy.key=value.

Example

Request:
{
 "method":"listPolicyAttributes",
 "params":{"volume_name":"policy_test_volume"},
 "jsonrpc":"2.0",
 "id":"1"
}
Response:
{
 "result":["1000.region=B","1000.country=DE"],
 "jsonrpc":"2.0",
 "id":"1"
}

JSON-RPC method: setPolicyAttribute

Description

Sets custom attribute filters on policies like 1000.country=DE or 1000.not.country=UK. Set the value to an empty string "" to remove an attribute.

See the XtreemFS user guide for more examples.

Syntax

setPolicyAttribute volume_name attribute_name attribute_value

Parameters

  • volume_name: the name of the volume to list attribute filters.
  • attribute_name: the policy followed by a dot and the name of the attribute to be set.
  • attribute_value: the value of the attribute to be set.

Return Value Format

String : status message.

Example: only use OSDs from country DE

Request:
{
 "method":"setPolicyAttribute",
 "params":{"volume_name":"policy_test_volume","attribute_name":"1000.country","attribute_value":"DE"},
 "jsonrpc":"2.0",
 "id":"1"
}
Response:
{
 "result":{
   "message":"Attribute xtreemfs.policies.1000.country set to DE on policy_test_volume",
   "usable_osds":3,
   "volume_name":"policy_test_volume"
  }
 "jsonrpc":"2.0",
 "id":"1"
}

Example: use all OSDs but from country UK

Request:
{
 "method":"setPolicyAttribute",
 "params":{
   "volume_name":"policy_test_volume","attribute_name":"1000.not.country","attribute_value":"UK"
  },
 "jsonrpc":"2.0",
 "id":"1"
}
Response:
{
 "result":{
   "message":"Attribute xtreemfs.policies.1000.not.country set to UK on policy_test_volume",
   "usable_osds":3,
   "volume_name":"policy_test_volume"
  }
}

JSON-RPC method: setACL

Description

Adds the user to the ACL on the current volume.

Syntax

setACL volume_name user_name user_accessrights

Parameters

  • volume_name: the name of the volume to set the ACLs for.
  • user_name: the user to be add to the ACLs.
  • user_accessrights: the new access rights of that user.

Return Value Format

String : status message.

Example

Request:
{
 "method":"setACL",
 "params":{"volume_name":"policy_test_volume","user_name":"patrick","user_accessrights":"xrw"},
 "jsonrpc":"2.0",
 "id":"1"
}
Response:
{
 "result":"Added user 'patrick' 'xrw' to ACL."
 "jsonrpc":"2.0",
 "id":"1"
}

JSON-RPC method: listACLs

Description

List the ACLs set on the current volume.

Syntax

listACLs volume_name

Parameters

  • volume_name: the name of the volume to list the ACLs for.

Return Value Format

Map<String,String> : map containing (user, access-rights)-pairs. access-rights contain r,w or x for read, write or access.

Example

Request:
{
 "method":"listACLs",
 "params":{"volume_name":"policy_test_volume"},
 "jsonrpc":"2.0",
 "id":"1"
}
Response:
{
 "result":{"g:":"rwx","patrick":"rwx","o:":"rwx","u:":"rwx"},
 "jsonrpc":"2.0",
 "id":"1"
}

JSON-RPC method: removeACL

Description

Removes the user from the ACL on the current volume.

Syntax

removeACL volume_name user_name

Parameters

  • volume_name: the name of the volume to change the ACLs for.
  • user_name: the user to be removed from the ACLs.

Return Value Format

String : status message.

Example

Request:
{
 "method":"removeACL",
 "params":{"volume_name":"policy_test_volume","user_name":"patrick"},
 "jsonrpc":"2.0",
 "id":"1"
}
Response:
{
 "result":"Removed user 'patrick' from ACL."
 "jsonrpc":"2.0",
 "id":"1"
}

JSON-RPC method: getServers

Description

Returns the list of ip-addresses of XtreemFS servers for the current volume.

Syntax

getServers volume_name

Parameters

  • volume_name: the name of the volume to list all servers for.

Return Value Format

List<String> : a list of unique ip-addresses of the XtreemFS servers. The first ip-address always corresponds to the DIR-address

Example

Request:
{
 "method":"getServers",
 "params":{"volume_name":"test_volume"},
 "jsonrpc":"2.0",
 "id":"1"
}
Response:
{
 "result":["localhost"]
 "jsonrpc":"2.0",
 "id":"1"
}

JSON-RPC method: startVINAgents

Description

Lets all GAFS/XtreemFS servers join a given VIN network.

This call requires the user's public ssh-key to be stored within the servers's ".ssh/authorized_keys"-file. 

Syntax

startVINAgents  server_ips vin_agent_identifiers ibis_pool_name ibis_server_address

Parameters

  • server_ips: the XtreemFS servers which have to start a VIN agent.
  • vin_agent_identifiers: the vin identifiers for each XtreemFS server.
  • ibis_pool_name: the ibis pool name.
  • ibis_server_address: the ibis server address.

Return Value Format

String[] : list of servers, which joined to a given VIN network.

Example

Request:
{
 "method":"startVINAgents",
 "params":{"server_ips":"localhost", "vin_agent_identifiers":"identifier", "ibis_pool_name":"pool_name", "ibis_server_address":"pool_address"},
 "jsonrpc":"2.0",
 "id":"1"
}
Response:
{
 "result":["localhost"],
 "jsonrpc":"2.0",
 "id":"1"
}

Using the gafs-manager

You can use any library to call the gafs-manager component using JSON-RPC calls. To call JSON-RPC from Java, we recommend the following framework:

Required components:

A simple Java example to call the gafs-manager interface looks as follows:

  int requestId = 0;

 // The GAFS-manager server URL
 URL serverURL = new URL("http://localhost:8080/xtreemfs-jsonrpc/executeMethod");

 // Create new JSON-RPC 2.0 client session
 JSONRPC2Session mySession = new JSONRPC2Session(serverURL);
   
 // Construct new request
 Map<String, String> parametersMap = new HashMap<String, String>();
 parametersMap.put("volume_name", "testVolume");

 // Note, that the password has to be configured by the admin_password-parameter
 // within the default_dir-config and can not be passed as a parameter anymore.
 //parametersMap.put("password", "");

 // construct a JSON-RPC request
 JSONRPC2Request req = new JSONRPC2Request(
       method,
       parametersMap,
       "id-"+(++requestId));

 // send the request to the server and get the response
 JSONRPC2Response res = mySession.send(req);

 if (res.indicatesSuccess()) {
    [...]
  }
 else {
    [...]
  }

See http://software.dzhuvinov.com/json-rpc-2.0-client.html for more details and examples.