Java Webservice API Client

Purpose

To help those who would like to automate queries, InterMine provides a client library to simplify access to our REST webservice API. This library is designed to facilitate access to information when you already know what you are looking for, or when you are doing lots of similar queries on a repetitive basis.

Prerequisites

The InterMine client package comes as a prepackaged JAR with all of its dependencies pre-packaged in. The only dependencies are thus:

• java (1.5+)
• javac - another compiler.

Installation

The client library can be downloaded as a zip file containing the required libraries. It can then be run from its unzipped location, or imported into an IDE such as Eclipse.

   wget http://www.intermine.org/lib/java-intermine-webservice-client-2.0.zip
   unzip java-intermine-webservice-client-2.0.zip

Downloading the Source

You can also download our source code. This requires checking out the InterMine source tree as a whole, as there are a number of dependencies - be aware that a source check-out is currently in the order of 2GB of size. For practical use, we strongly recommend the use of the prepackaged library, which weighs in at only around 2.5MB (most of which is the prepackaged dependencies).

Also, be aware that the code from the trunk may not always be as thoroughly tested or bug-free as code from the latest release, or that available from your mine.

From the InterMine svn repository:

   svn co svn://subversion.flymine.org/trunk/flymine

Documentation

We provide thorough API documentation at http://www.intermine.org/docs/java-client-docs

Querying A Webservice

The two central concepts we use in this library are:

• Services - which represent connections to a resource
• Queries - which represent your request for data

The basic procedure for performing webservice requests is to:

1. Connect to a service.
2. Define a query.
3. Pass it to the service which returns results.

Connecting to A Service

Access to the different resources of a webservice is mediated through the ServiceFactory convenience class. To construct a ServiceFactory all you need is the address of the root of that service:

ServiceFactory flymine = new ServiceFactory("http://www.flymine.org/query/service");

From this factory you can obtain connections to the individual resources an InterMine webservice provides:

// The following services are the ones you will use most often
ModelService modelService             = flymine.getModelService();    // For retrieving the data model
QueryService queryService             = flymine.getQueryService();    // For making requests for data using queries
TemplateService templateService       = flymine.getTemplateService(); // For making requests for data using templates

// The following services are used less frequently
ListService listService               = flymine.getListService();     // For finding out which lists an object is in
AvailableTemplateService aTempService = flymine.getAvailableTemplateService() // For finding out which templates you have access to

Obviously, you should change the URL in the example above to that of the service you wish to query. Some examples of webservices that support this API include:

FlyMine

 http://www.flymine.org/query/service

RatMine

 http://ratmine.mcw.edu/ratmine/service

YeastMine

 http://yeastmine.yeastgenome.org/yeastmine/service

modMine

 http://intermine.modencode.org/query/service

MetabolicMine

 http://www.metabolicmine.org/beta/service

Defining A Query

Queries can either be instances of:

• org.intermine.pathquery.PathQuery - for arbitrary queries defined in your code.
• org.intermine.api.template.TemplateQuery - for predefined queries stored on the web service.

PathQueries

A Query has the following main concepts:

• The model - the data model of the web service, used to ensure we construct a valid query.
• The view - the output columns that will appear in the results table
• The constraints - the filters on the results that specify what we want. (optional)

All queries are for a list of a certain type of object - thus all 'paths' in the query must descend from the same root - so Gene.pathways.name and Gene.organism.name are ok in the same query, but Pathway.name and Organism.name are not, as the query does not know how the organism and the pathway are linked.

Optionally, you can also specify the ways the results are sorted, (the sort order) and the ways the rules are combined (the constraint logic).

Thus defining a PathQuery has the following form:

1. Fetch a model, and use this to construct a new query object:

   Model model = modelService.getModel();
   PathQuery query = new PathQuery(model);
   

2. Define the output columns, and potentially determine which one we want to sort the result set by:

   query.addViews("Gene.symbol", "Gene.pathways.name", "Gene.organism.name", "Gene.length");
   // The default sort-order is the first view specified - here Gene.symbol
   // We can group by organism by using that as the sort order:
   query.addOrderBy("Gene.organism.name", OrderDirection.ASC);
   

3. Define the results we want by setting up filters, ie. constraints on the query - here we only want Genes involved in phosphate pathways of some kind, in Drosophilidae:

   //The * wildcard can be used to make basic matching patterns
   String code = query.addConstraint(Constraints.eq("Gene.pathways.name", "*Phosphate*"));
   String code = query.addConstraint(Constraints.eq("Gene.organism.genus", "Drosophila"));
   
   // The default interaction between constraints is for them to be cumulative - ie:
   // we assume "A and B". This can be specified:
   query.setConstraintLogic("A and B");
   

PathQueries have to be passed to a QueryService to retrieve results. This is done with a call to getAllResults(). Below you can see an example of fetching results and printing them out:

List<List<String>> result = service.getAllResults(query);
    System.out.println("Results:");
    for (List<String> row : result) {
        for (String cell : row) {
            System.out.print(cell + " ");
        }
        System.out.print("\n");
    }
}

Results are normally returned as a multi-dimensional list (a list of rows, which are lists of Strings). In this default format, all result columns are returned in their String representation, be it "D. melanoaster" or "0.1234". To return typed results (and get an int or a double where the result is an int or a double) you should use the alternative result format getAllJSONResults(). An example of using this to get typed results that can be arithmetically manipulated can be seen below:

List<JSONObject> genes = service.getAllJSONResults(query);
System.out.println("Results:");
int total = 0;
for (JSONObject gene : genes) {
    System.out.println("Name: " + gene.getString("name"));
    System.out.print("Pathways: "); 
    JSONArray pathways = gene.getJSONArray("pathways");
    for (int i = 0; i < pathways.length(); i++) {
        JSONObject pathway = pathways.getJSONObject(i);
        System.out.print(pathway.getString("name"));
        if ((i + 1) < pathways.length()) {
            System.out.print(",");
        }
    }
    total += gene.getInt("length");
    System.out.print("\n");
}
System.out.println("Average length: " + (total/genes.size()));

Templates

Templates are predefined queries stored on the server and accessed by name. The advantages of templates are generally in concision, since they do not need to be defined before use, and maintainability, since if you access public templates they will be updated for you. If you choose to access your own private templates (see below) you also benefit by being able to run queries you have built using the graphical interface.

For templates, we do not need to construct a new TemplateQuery, but instead we refer to the template we want to run by its name and parameters. For example to run the template that finds pathways for a specific gene in FlyMine I need:

• Its name: Gene_Pathway. You can find this in the browser address bar when you are looking at a template page ( example)
• The parameters it takes. The Gene_Pathway template takes a single constraint: the gene whose pathways we want to find.

To define the parameters, we construct a list of them:

List<TemplateParameter> parameters = new ArrayList<TemplateParameter>();
// The first element of a Template parameter (the path - here "Gene") must be the same as the one on the query.
// You can change the operation and the values to anything compatible with the path.
parameters.add(new TemplateParameter("Gene", "LOOKUP", "bsk", "D. melanogaster"));

To fetch results from the web service we require a TemplateService on which we call getAllResults(). Parsing the results takes exactly the same form as for query results, and we can also call getAllJSONResults() for templates as well.

List<List<String>> result = service.getAllResults(templateName, parameters);
    System.out.print("Results: \n");
    for (List<String> row : result) {
        for (String cell : row) {
            System.out.print(cell + " ");
        }
        System.out.print("\n");
    }
}

Logging In to Access your Private Templates And Lists

To access your private templates and lists you need to provide your login information by using setAuthentication(java.lang.String, java.lang.String). Eg.:

service.setAuthentication("YourUserName", "YourPassWord");

Your user name is usually the email address you signed up with.

Further Examples

You do not have to write your own code - you can get the webapp for your preferred webservice to generate the code for you - simply click on one of the code generation links at the bottom of any query-builder or template-form page. This will produce code you can immediately run, and perhaps later edit to refine the query:

Other clients

We also have client libraries for:

• Perl
• Python
• JavaScript