Contents¶

Configuring MOTECH through the UI¶

If you’ve selected the UI as the configuration mode, the following screen will be displayed.

Here you are able to select the language MOTECH will be using, as well as one of the following login mode:

• Repository Using this mode will result in creating accounts locally.
• OpenID Using this mode will result in using the OpenID standard. For more information about the OpenID, please visit the www.openid.net website.

Configuring MOTECH through FILE¶

If you’ve selected this method of configuring MOTECH you’ll have to provide the configuration file. The file must have the .properties extension and must be placed in the config subfolder of the MOTECH folder(most likely /home/{username}/.motech/config) and must store the following properties.

system.language

The language the MOTECH will be using. The accepted values are en, pl etc.

login.mode

The login mode that will be used. The accepted values are repository and openid.

system.language=en
login.mode=openid
provider.name=UbuntuOne
provider.url=https://login.launchpad.net/

First login through repository¶

If you’ve chosen repository as the login mode, the following window will be displayed. Here you’ll have to enter the credentials you provided in the MOTECH configuration step.

First login through OpenID¶

If you’ve chosen OpenID as the login mode, the following window will be displayed. Here, you’ll have to click on the Sign in with Provide name button, which will result in redirecting you to the OpenID provider login page, where you’ll have to enter valid credentials for that provider.

Default Behavior¶

By default, all configuration files are loaded from one of the following locations:

....${user.home}/.motech/config/

For example, if Motech runs under the user motech in Linux, configuration files will be searched in “/home/motech/.motech/config” folder.

Or:

..../etc/motech/

If the configuration files are not found in the above mentioned location, files will be searched in “/etc/motech” folder.

Overriding Default Behavior¶

If you want to override the location where configuration files are loaded from, you have to add config-locations.properties to ${user.home}/.motech or to ${user.home}/ directory. Otherwise the property file in classpath will be picked up. Default config-locations.properties file:

....config.location= ${sys:user.home}/.motech/config/,/etc/motech/.

where “${sys:user.home}” is used to specify the home directory.

Note that, multiple locations can be specified for config.location property, and motech-settings.properties (which contains platform core config) config file will be searched starting from the first location and then falling back to next specified location, if it is not found. The directory in which it is found, is considered as the current config location and all other config files will be looked only in that particular location. For e.g., in the above sample config, if you have motech-settings.properties in /etc/motech/, then all config files will be searched in /etc/motech/ location only. You cannot have files in different locations.

Case 1: When ConfigSource is FILE¶

Define motech-settings.properties file in any one of the locations defined in config-locations.properties with the above mentioned properties.

Case 2: When ConfigSource is UI¶

After server startup, if core settings are not configured already, you will be presented with a startup page which asks for System Language, Queue URL for events, Login Mode and user setup based on login mode. Other activemq settings can be changed in Settings tab after logging in.

Case 1: When ConfigSource is FILE¶

Module specific property files can be added to:

....<config-location-dir>/<module-symbolic-name>/ directory

and any JSON templates/configurations to:

....<config-location-dir>/<module-symbolic-name>/raw/ directory.

A typical example of a motech’s module symbolic name:

....<module-name>

prefixed with “org.motechproject.motech-”.

All these files are monitored for changes. So, any change to these config files at runtime would be detected and saved in DB. Restart the module if required using Manage Modules tab in UI. We are enhancing the config monitor to raise an event in case of config change. This event can be listened by interested modules and take appropriate actions.

Case 2: When ConfigSource is UI¶

After server startup, you can find each module having settings UI associated with it in the Manage Modules tab, where you can edit the properties for the module. Also, restart the module if required. We are enhancing the config monitor to raise an event in case of config change.

MDS generated entities bundle¶

All classes generated by MDS live in the mds-entities OSGi bundle, which gets generated at runtime and installed in the directory ~/.motech/bundles. The bundle is always regenerated when changes are made to the MDS schema. This generated bundle can also be downloaded using the following url: http://<motech_url>/modulemds/jar.

Automatically added fields¶

All entities in MDS will be enhanced with the following fields automatically:

Access to these fields can be done through reflections, through re-declaring them in the DDE class or by inheriting the MDSEntity class.

Creating EUDE through UI¶

The easiest way to create EUDE entities is to use the MOTECH UI. First select Data Services from the left navigation menu(Modules menu), then navigate to the Schema Editor tab. You will see a dropdown allowing to select an existing entity for modification or deletion. Next to the dropdown menu you will see a New Entity button.

After that the user is asked for the name of the entity. This can be anything that is a legal name of a class in Java.

The view for managing entity fields is then displayed to the user. Users can add a field by selecting its type, choosing a name and a display name. ‘display name’ represents what will be visualised to the users in the MDS Data Browser, task editor etc. ‘name’ represents the actual name of the field that will be used for class and table creation. After this data is entered, hitting the green plus sign will add the field.

The field is then expanded and the user is presented with options to modify the field settings:

The Basic sections allows to change the previously entered name and display name, it also allows marking the field as required, meaning that users will be prevented from creating an instance without any value in this field. A default value for the field can also be entered, as well as a tooltip that will be shown to users creating instances of the entity.

The Metadata section allows adding metadata to the field. This used internally by MDS for features such as relationships. End users should not worry about this section, but advanced users can add any values they wish for their own processing needs. Metadata is retrieved with the field schema using the Entity API. An example of using metadata could be a scenario when we are writing a third party export tool, that takes the MDS Schema and imports it into a 3rd party system. The field metadata can be used by that tool in order to recognize some fields as requiring special processing logic.

The Validation section allows setting specific validation rules for the field. Users will then be constrained by these validations when creating instances of the entity. Validations are type specific.

The Settings tab allows users to set type specific settings of the field. An example setting is the ‘Max text length’ of a String field, which indicates the maximum length of the string at the database level.

Existing fields can be deleted using the trash bin icon next to their type.

When the user is done modifying the entity, clicking Save changes will save the changes to schema and regenerate MDS entities. Clicking Abandon Changes will abandon all changes made by the user since the last save.

Defining a Lookup through the UI¶

Users can use the UI for adding lookups to an entity. These lookups can then be executed either directly through the data services or using the Data Browser UI. In order to add a new lookup, first open the advanced settings of an entity by clicking the ‘Advanced Settings’ button.

After that users can create lookups by clicking on the ‘New Lookup’ button.

The name fo the lookup can then be modified as well as whether it returns a single or multiple objects. In order to make a lookup useful, it has be executed on a given set of fields, which can be added on the right side of the window by clicking the ‘New Lookup Field’ button and selecting the right field from the dropdown. They can be deleted using the trash bin button.

In order to remove a lookup, the delete button in the lower right of dialog can be used.

When the user is done adding lookups to an entity, clicking Save changes will save the changes and trigger regeneration. Clicking Abandon Changes will abandon all changes made by the user since the last save.

Creating EUDE through the Entity API¶

Creation of entities can be also done using the org.motechproject.mds.service.EntityService. This an OSGi service exposed by MDS which allows creation and modification of MDS entities, exposing everything that the UI does. In order to use the service it has to be retrieved from the OSGi context, either directly using the OSGi API or a Blueprint reference can be used to inject a proxy for that service directly as a Spring bean.

Example of retrieving the service manually:

import org.motechproject.mds.service.EntityService;
import org.osgi.framework.*;

...

public EntityService getEntityService() {
    // note that if using Spring, the BundleContext can be injected as any other bean
    // which allows skipping this step
    BundleContext bundleContext = FrameworkUtil.getBundle(EntityService.class).getBundleContext();

    // get the service reference from the bundle context
    ServiceReference<EntityService> ref = bundleContext.getServiceReference(EntityService.class);

    // return the service for the reference, or null if there are no references
    // the service should always be available, so a null reference definitely indicates some sort error
    return ref == null ? null : bundleContext.getService(ref);
}

and the preferred way using blueprint. Note that thanks to this declaration an EntityService bean becomes available in your Spring context.

<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans"
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xmlns:osgi="http://www.eclipse.org/gemini/blueprint/schema/blueprint"
    xsi:schemaLocation="http://www.springframework.org/schema/beans
        http://www.springframework.org/schema/beans/spring-beans.xsd
        http://www.eclipse.org/gemini/blueprint/schema/blueprint
        http://www.eclipse.org/gemini/blueprint/schema/blueprint/gemini-blueprint.xsd">

    <osgi:reference id="entityService" interface="org.motechproject.mds.service.EntityService"/>

</beans>

After getting hold of the service the entity can be created using the createEntity method:

EntityService entityService = getEntityService();

EntityDto entity = new EntityDto("Patient");

// the EntityDto instance returned will have the id value set
entity = entityService.createEntity(entity);

If we want to edit an existing entity, we can retrieve it using the EntityService:

// We can use the org.motechproject.mds.util.ClassName utility in order
// to get the EUDE class name given just the name
String className = ClassName.getEntityName("Patient");

// className is org.motechproject.mds.entity.Patient
EntityDto entity = entityService.getEntityByClassName(className);

When we have the EntityDto instance, fields can get added to the entity using the service and EntityDto returned:

// a simple integer field
FieldDto simpleField = new FieldDto("simpleInt", "Simple integer", TypeDto.INTEGER);

// a required name field
FieldDto nameField = new FieldDto("name", "Patient Name", TypeDto.STRING, true);

// an optional date of birth field, with a tooltip
FieldDto dobField = new FieldDto("dob", "Date of Birth", TypeDto.DATETIME, false, null,
        "Patients date of birth, leave blank if unknown");

// a required Social ID field, defaulting to 0
FieldDto socialIdField = new FieldDto("socialId", "Social ID", TypeDto.LONG, true, 0L);

// add the fields to the entity created earlier
entityService.addFields(entity, simpleField, nameField, dobField, socialIdField);

In order to make these changes take effect, data bundle regeneration must be triggered.

Creating Lookups through the API¶

Just as any other edits on the entity schema, lookups can also be created using the EntityService. In a similar fashion to fields, the addLookups method can be used for adding lookups to an entity. Given that we have the EntityDto object and the EntityService(), we can create lookups in the following manner:

// this lookup will check the name field, during an exact comparison
LookupDto lookupByName = new LookupDto("By name",
        true, // single object return
        true, // expose this lookup through REST
        Arrays.asList(new LookupFieldDto("name", LookupFieldDto.Type.VALUE)
));

// this a complex lookup using multiple fields
LookupDto complexLookup = new LookupDto("Complex lookup",
        false,  // return multiple objects
        false,  // do not expose by REST
    Arrays.asList(
        // the custom operator matches() will be used for querying on the name field
        new LookupFieldDto("name", LookupFieldDto.Type.VALUE, Constants.Operators.MATCHES),
        // the dob parameter will take a range, with a min and max value
        new LookupFieldDto("dob", LookupFieldDto.Type.RANGE),
        // for the state field, a set of possible values can be supplied
        new LookupFieldDto("state", LookupFieldDto.Type.SET),
        // the search through relationship fields is possible using the dot operator
        new LookupFieldDto("relationshipField.number", LookupFieldDto.Type.VALUE))
);

// add the lookup
entityService.addLookups(entity, lookupByName, complexLookup);

In order to make these changes take effect, data bundle regeneration must be triggered.

Regenerating the entities bundle¶

After we are done with modifications to the entity schema, we must trigger regeneration in order for the classes to get updated and made available in OSGi. For this we need to use org.motechproject.mds.service.JarGeneratorService, which we can retrieve the same way that we can retrieve the EntityService. Once we have an instance of the service, all we need to do is call the regenerateMdsDataBundle method:

JarGeneratorService jarGeneratorService = getJarGeneratorService();

jarGeneratorService.regenerateMdsDataBundle();

After the schema gets regenerated and all bundles using MDS get refreshed, the EUDE class should be available for use.

Programmatic access to EUDE entities¶

EUDE classes can be accessed using java reflections. This is an example of creating an instance using reflections:

// first get the interface class name of the name entity
// this helper method will always return org.motechproject.mds.entity.Patient
String interfaceName = ClassName.getInterfaceName("Patient")

// Retrieve the Data Service
MotechDataService service = ServiceUtil.getServiceForInterfaceName(bundleContext, interfaceName);

// Get the Class object for the entity
Class entityClass = service.getClassType();

// create a patient instance and set the name to "John"
Object instance = entityClass.newInstance();
PropertyUtil.setProperty(instance, "name", "John");

// save it using the service
service.create(instance);

As you can see the access is done through the Data Service. We can obtain the Class object for the generated class and use it for doing all required operations using reflections.

Defining entities - the @Entity annotation¶

In order to define a DDE by using the org.motechproject.mds.annotations.Entity annotation. This are the contents of Patient.java, an example fo a DDE entity:

package org.motechproject.example;

import org.motechproject.mds.annotations.*;

@Entity
public class Patient {

}

When the module containing this entity gets installed MDS will scan it for classes annotated with @Entity, and the class above would get picked up for processing. Schema for the entity is then generated and persisted in the database of MDS, the class is also enhanced by DataNucleus. The MDS weaving hook then replaces the bytecode for this class in module ClassLoaders with the DataNucleus/MDS enhanced version, making it available to the modules using it.

The @Entity annotation has the following parameters:

• name - The name of the entity displayed to the user. Defaults to the simple name of the annotated class.
• module - The name of the module for this entity. Defaults to the module name of the bundle from which this entity comes from.
• namespace - The namespace in which the entity is defined. Optional, defaults to empty.
• tableName - The actual name of the table in the database for this entity. Allows users to directly control the name in the data store. The default table name will take the form of: MDS_<MODULE>_<NAMESPACE>_<ENTITY_NAME>. If an entity has no namespace or module, those parts will be omitted.
• recordHistory - Set to true if MDS should record history for this entity.

DDE entity fields - @Field and @Ignore annotations¶

An entity does not have much use without any fields. MDS will treat any public field or field with public getter/setter in the class as an MDS field. In the class below, the field name will be picked up automatically as a field to be persisted in the database:

@Entity
public class Patient {

    private String name;

    public String getName() {
        return name;
    }

    public void setName(String name) {
        this.name = name;
    }
}

The @Field annotation can be used for more explicit marking and control over the fields basic properties. In the example below, the required parameter of the annotations is used to mark the name field as required, moreover the physical column name in the database is set to “P_NAME”:

@Entity
public class Patient {

    @Field(name = "P_NAME", required = true)
    private String name;

    public String getName() {
        return name;
    }

    public void setName(String name) {
        this.name = name;
    }
}

The @Field annotation could also be placed on the setter or getter methods for the same effect.

Not every public field, or not every field that has a public getter or setter has to be persisted in the database. The @Ignore annotation can be used for marking such field as not persistent:

@Entity
public class Patient {

    @Ignore
    public String name;
}

The name field in the example above will not become a database field and no MDS schema will be generated for it. This field will also not be accessible through the data browser.

DDE relationships¶

Creating relationships between entities is currently only possible for DDE. The definition of a relationship depends on the type of the relation. MDS supports one-to-one, one-to-many, many-to-many and master-detail relationships, both uni-directional and bi-directional. The way to define relationships for DDEs is presented in the examples below.

• One-to-one To create a one to one relationship, one of the related entities, should define a field of class, that represents the second entity. Both classes must of course be valid MDS Entities. The code below, provided that Book is an entity, will create a simple, uni-directional, one-to-one relationship between Author and Book.

@Entity
public class Author {
    @Field
    private String name;

    @Field
    private Book book;

    ...
}

• One-to-many To create a one to many relationship, one of the entities should define a collection of related entity. Just like in one-to-one relationships, both classes must be valid MDS entities to work. The code below shows an example of a simple, uni-directional, one-to-many relationship between Author and Book (one author is related with many books).

@Entity
public class Author {
    @Field
    private String name;

    @Field
    private Set<Book> book;

    ...
}

• Bi-directional relationships The bi-directional relationship is a model, in which both sides of a relation are aware of the existence of a relationship and can both refer to the other side of a relation.

  To make the relationship bi-directional, two additional steps must be taken:

  • The second entity must also define a relationship to the other entity
  • Exactly one MDS field of a bi-directional relationship must be annotated with the @javax.jdo.annotations.Persistent(mappedBy = “fieldName”) annotation. The fieldName should correspond to the field name that is in a relationship, in the another entity.

  Please see the code below, for an example of a one-to-many, bi-directional relationship.

@Entity
public class Author {
    @Field
    private String name;

    @Field
    @Persistent(mappedBy = "author")
    private Set<Book> book;

    ...
}

@Entity
public class Book {
    @Field
    private String title;

    @Field
    private Author author;

    ...
}

• Many-to-many Mds supports two types of many to many relationship. First type is M-N Set relation which is bi-directional, if you need more information you should read datanucleus M-N Set relation documentation. The code below shows an example of a many-to-many set relationship.

@Entity
public class Author {
    @Field
    private String name;

    @Field
    @Persistent(mappedBy = "author")
    private Set<Book> book;

    ...
}

@Entity
public class Book {
    @Field
    private String title;

    @Field
    private Set<Author> author;

    ...
}

Second type is M-N Indexed Lists relation which is modelled as 2 1-N unidirectional relations using join tables. Very important is to use the @IndexedManyToMany annotation at both ends of the relation instead of the @Persistent(mappedBy = “fieldName”). If you need more information you should read datanucleus M-N Indexed List relation documentation. The code below shows an example of a many-to-many indexed list relationship.

@Entity
public class Actor {

    @Field
    private String name;

    @Field
    @IndexedManyToMany(relatedField = "actors")
    private List<Movie> movies;

    ...
}

@Entity
public class Movie {

    @Field
    private String name;

    @Field
    @IndexedManyToMany(relatedField = "movies")
    private List<Actor> actors;

    ...
}

The code below shows an example how to properly use many-to-many indexed list relationship.

Actor actor1 = actorDataService.findByName("actor_1");
Actor actor2 = actorDataService.findByName("actor_2");
Movie movie = movieDataService.findByName("movie");

movie.getActors().add(actor1);
movie.getActors().add(actor2);
actor1.getMovies().add(movie);
actor2.getMovies().add(movie);

movieDataService.update(movie);

• Master-detail MDS also supports master-detail model, where entity can inherit some fields from another entity. This is achieved by simple class inheritance, using Java keyword extends. Naturally, both classes must be valid MDS entities for this to work. The code below shows an example of such master-detail model.

@Entity
public abstract class Config {
    @Field
    private String name;

    @Field
    private Map<String, String> properties;

    ...
}

@Entity
public class ModuleConfig extends Config {
    @Field
    private String moduleName;

    @Field
    private String moduleVersion;

    ...
}

• Eager/lazy loading By default loading an entity with relationship will load its related entities, but that behaviour can be configured through @Persistent(defaultFetchGroup = “true/false”) annotation. Please see the code below for an example.

  @Entity
  public class Author {
  
      @Field
      private String name;
  
      @Field
      @Persistent(defaultFetchGroup = "false")
      private Set<Book> books;
  
      ...
  }
  

  By defining class this way the set of books won’t be fetched from the database unless it is explicitly said (e.g. by calling getBooks() method on object of the Author class) to. This approach simplifies the queries sent to the database and lower its overall usage.

  Lets take a look at the following example using Subscriber and Subscription classes.

  @Entity()
  public class Subscriber extends MdsEntity {
  
      @Field
      private Long callingNumber;
  
      @Field
      @Persistent(mappedBy = "subscriber")
      private Set<Subscription> subscriptions;
  
      ...
  
  }
  

  @Entity
  public class Subscription extends MdsEntity {
  
      @Field
      private String subscriptionId;
  
      @Field
      private Subscriber subscriber;
  
      ...
  
  }
  

  With the default approach query responsible for fetching subscriptions will look like this

  SELECT 'entity.class.name.Subscription' AS
      NUCLEUS_TYPE,
      A0.creationDate,
      A0.creator,
      A0.id,
      A0.modificationDate,
      A0.modifiedBy,
      A0.owner,
      A0.subscriber_id_OID,
      A0.subscriptionId
  FROM MOTECH_PLATFORM_DATA_SERVICES_TEST_BUNDLE_SUBSCRIPTION A0
      WHERE EXISTS (
          SELECT 'entity.class.name.Subscriber' AS
              NUCLEUS_TYPE,
              A0_SUB.id AS DN_APPID
          FROM MOTECH_PLATFORM_DATA_SERVICES_TEST_BUNDLE_SUBSCRIBER A0_SUB
              WHERE A0.subscriber_id_OID = A0_SUB.id)
  

  which gets simplified to

  SELECT 'entity.class.name.Subscription' AS
      NUCLEUS_TYPE,
      A0.creationDate,
      A0.creator,
      A0.id,
      A0.modificationDate,
      A0.modifiedBy,
      A0.owner,
      A0.subscriber_id_OID,
      A0.subscriptionId
  FROM MOTECH_PLATFORM_DATA_SERVICES_TEST_BUNDLE_SUBSCRIPTION A0
      WHERE A0.subscriber_id_OID = 1
  

  if we remove Subscriptions field from the default fetch group (by adding @Persistent(defaultFetchGroup = "false") annotation to the :code:`Subscriptions field. This query requires one table scan less and won’t be sent to the database unless explicitly ordered to.

Using DataNucleus annotations¶

DataNucleus JDO annotations can be used for enhancing DDEs. These annotations will be taken into consideration by DataNucleus and override the metadata that MDS generates. For example the @javax.jdo.Unique annotation can be used in order to mark fields in an entity as unique. Refer to the DataNucleus documentation for more information on using those annotations.

DDE service interfaces¶

DDEs can define their own interfaces that extend the default service interface that will be used for generating MDS services. The service will be published under that interface, and thanks to inheritance, it will also expose type safe methods from the base service. Here is an example of defining an interface for a ‘Patient’ DDE:

public interface PatientDataService extends MotechDataService<Patient> {

}

Thanks to this declaration type safe access to methods of the interface will be gained, the generic parameter Patient will be inserted for the returned/parameter values.

This way of defining services for DDEs also allows to define additional lookups on the service. These lookups are defined as plain method declarations with annotations and their implementation will be generated at runtime by MDS. The lookup method must be annotated with a @Lookup annotation. Method parameters should be marked with @LookupField annotation in order to connect the parameter with the actual entity field.

public interface PatientDataService extends MotechDataService<Patient> {

    /*
     * This lookup finds a single patient based on the field 'name'.
     * So invoking this method like this: byName("John") will
     * return the patient with the name "John".
     */
    @Lookup
    Patient byName(@LookupField(name = "name") String name);

    /*
     * The count method. Note that if this method is not defined,
       it will be generated automatically from the lookup above.
     */
    long countByName(String name);

     /*
     * Same as above, but returns multiple results.
     */
    @Lookup
    List<Patient> byName2(@LookupField(name = "name") String name);

    /*
     * Same as above, but with QueryParams. Note that if this method is not defined,
       it will be generated automatically from the lookup above.
     */
    @Lookup
    List<Patient> byName2(@LookupField(name = "name") String name, QueryParams queryParams);
}

The type of the parameter must match the type of the field, unless its one of the two special types:

Range - ranges can be used for looking up values that fall within the given range. An example is a range of dates. Range consist of min and max values, it is possible to provide only one of these values so there will be no boundary on the second end.

public interface PatientDataService extends MotechDataService<Patient> {

    /*
     * Looks up patients for which the date of birth falls in the supplied range of
     * values. Example of usage:

        byDateOfBirth(new Range<>(DateTime.now().minusYears(30), DateTime.now().minusYears(10)));

     * this returns patients born between 30 and 10 years ago.
     */
    @Lookup
    List<Patient> byDateOfBirth(@LookupField(name = "dob") Range<DateTime> dobRange);

}

Set - Doing lookups by sets is also possible. Instead of providing a single value, you provide a set of values. If an instance field matches one of the values, that is considered a hit(basically this is logical OR matching).

public interface PatientDataService extends MotechDataService<Patient> {

    /*
     * Looks up patients which name matches one of the values from the set.
     * Usage example:
     *
     *  byName(new HashSet<>(Arrays.asList("Tom", "John", "Bob")));
     *
     * This will return patients named Tom, John or Bob.
     */
    @Lookup
    List<Patient> byName(@LookupField(name = "name") Set<String> names);

}

Lookups can also use custom operators. The operator is inserted between the field name and the lookup parameter in the JDO query generated for the lookup. The default symbol is ‘=’ - the equality sign, however different operators can also be used. Both JDO QL operators and methods can be used for lookups. If an operator like “<” is provided as the custom operator, it will be put between field name and parameter value. If the operator has the form a function like “matches()” it will generate a method call of the form “parameter.matches(value)” - the value is inserted between the brackets. In order to provide a custom operator for a lookup field, the customOperator field of the @LookupField annotation has to be set:

public interface PatientDataService extends MotechDataService<Patient> {

    /*
     * Does a matches() lookup on the name field.
     * Because matches() is used, a regex pattern can be passed as the parameter.
     */
    @Lookup
    List<Patient> byName(@LookupField(name = "name", customOperator = "matches()") String name);

}

Defining editable lookups for DDE entities¶

One way to define lookups for DDE entities is to include a mds-lookups.json file in module resource directory. The file should be a valid array of EntityLookups class objects. Every lookup defined in the file will be added only once, so even after user had deleted lookup it won’t be recreated during module or MOTECH restart. This gives the user complete control over those lookups without any restrictions. The unique identifier of every lookup is its entity class name and lookup name combination. This is the intended way for modules to define lookups that should be made editable by the end user. Backend code should not depend on these lookups.

Example mds-lookups.json file.

[
    {
        "entityClassName" : "org.motechproject.tasks.domain.Task",
        "lookups" : [
            {
                "lookupName" : "Find Task by Owner",
                "singleObjectReturn" : false,
                "exposedViaRest" : false,
                "lookupFields" : [
                    {
                        "name" : "owner",
                        "type" : "VALUE",
                        "customOperator" : "\u003d\u003d",
                        "useGenericParam" : false
                    }
                ],
                "readOnly" : false,
                "methodName" : "findTaskByOwner",
                "fieldsOrder" : [
                  "owner"
                ]
            }
        ]
    }
]

Including the example json in Tasks module will result in adding lookup for Task entity that will return all tasks that are owned by the specified user.

Programmatic usage of DDE entities¶

All that has to be done in order to use a DDE is to retrieve the service for its interface. Because of the nature of DDEs, their classes are available during compile time. The service reference can be then retrieved using the standard OSGi facilities:

public PatientService getPatientService() {
    BundleContext bundleContext = FrameworkUtil.getBundle(Patient.class).getBundleContext();
    ServiceReference<PatientService> ref = bundleContext.getServiceReference(PatientService.class);
    return ref == null ? null : bundleContext.getService(ref);
}

The preferred way however is to use Blueprint OSGi references. The service will be injected as a Spring bean into the Spring application context of the module and can be then used as any other bean(for example it can be @Autowired into other beans).

<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans"
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xmlns:osgi="http://www.eclipse.org/gemini/blueprint/schema/blueprint"
    xsi:schemaLocation="http://www.springframework.org/schema/beans
        http://www.springframework.org/schema/beans/spring-beans.xsd
        http://www.eclipse.org/gemini/blueprint/schema/blueprint
        http://www.eclipse.org/gemini/blueprint/schema/blueprint/gemini-blueprint.xsd">

    <osgi:reference id="patientDataService" interface="org.motechproject.example.PatientService"/>

</beans>

Once the service instance is obtained, the only thing left to do is to just call the right method exposed.

Record versioning and optimistic locking¶

DDE entities support versioning of records. A version will be automatically increased after each update. This feature is useful when other user or other thread is working with the same record. When user performs update but record has been changed before this update an optimistic exception will be thrown. To enable versioning for the entity developer can extend MdsVersionedEntity or can use the @Version annotation. The following example shows how to use the annotation.

@Entity
@Version(strategy = VersionStrategy.VERSION_NUMBER, column = "version",
    extensions={@Extension(vendorName = "datanucleus", key="field-name", value="version")})
public class VersionedEntity {

    @Field
    private Long version;
}

Extending DDEs through the UI¶

Extending DDEs through the UI is not different from manipulating the schema of EUDE entities. Refer to the documentation section on creating EUDE entities for more info. In order to extend a DDE first go the MDS Schema Editor and select the DDE entity you wish to edit:

Next add the field you wish to add to the entity:

You can also add lookup to the DDE:

Finally, save your changes to trigger MDS schema regeneration and make your changes take effect(you can also abandon your changes if you wish):

Extending DDEs through code¶

Extending DDEs through code is no different from extending EUDE entities. The only difference is that the EntityDto for the DDE has to be retrieved by providing its class name. Refer to the documentation on extending EUDE through code.

Map type¶

You can declare map with keys and values having generic type. MDS supports the following types of generics :

• key types (String, Integer, Long)
• value types (String, Integer, Long)

If you use the supported types, the field will be stored as a separate table in a database. Otherwise the field will be serialized.

Controlling whether to record history¶

By default MDS doesn’t keep track of the instance revisions. Most of the DDEs that come with MOTECH modules have the tracking of the history disabled as well. To enable history tracking for the...

• Developer Defined Entity (DDE) - You have to set the recordHistory parameter of the @Entity annotation to true.

@Entity(recordHistory = true)

• End User Defined Entity (EUDE) - The Enable history audit option is available under the Advanced window of an entity, in the Auditing & Revision Tracking tab

  

Retrieving history using code¶

MDS exposes an implementation of the org.motechproject.mds.service.HistoryService. To make use of it, you should simply create a reference to that service in your blueprint:

<osgi:reference id="historyServiceOSGi" interface="org.motechproject.mds.service.HistoryService" />

From now on, you will be able to use the history service, just like any other Spring bean, for example, by placing the @Autowired annotation on a field of type org.motechproject.mds.service.HistoryService. The service allows recording history, deleting the whole history for an instance and retrieving the historical revisions of an instance.

Using Trash using code¶

Similar to the HistoryService mentioned above, MDS also exposes the TrashService that allows operations on the Trash bin from the code. To use the exposed service, create a reference in your blueprint file:

<osgi:reference id="trashServiceOSGi" interface="org.motechproject.mds.service.TrashService" />

Accessing the service also works the same way as with the HistoryService - treat it as any other Spring bean, for example by placing the @Autowired annotation on the field of type org.motechproject.mds.service.TrashService. The trash service allows to place instances in trash, retrieve instances from trash, schedule the trash purging, empty the trash and check current data retention settings.

Changing the settings through the UI¶

To change the data browsing settings via UI, go to the Schema Editor and select an entity for which you wish to set the settings. Go to the Advanced view and pick the Data Browsing tab. The first section, called Display fields, contains two tables. The table to the right shows fields that have been selected to display by default. The table to the left shows all other fields. The order of the fields in the Fields to display table corresponds to the order of the fields in the data browser UI. You can move fields from one table to another and change their order, using provided buttons, or by dragging the fields to their destination. The second section, named Filters allows to pick fields, for which the data browser UI will generate filters. Please note that only fields of a certain types will be displayed. The filters are generated automatically and are adjusted to the field type. For example, for the date types, there will be an option to set a filter for today, this week, this month and this year, while for boolean, this will be only true and false. When you finish making the changes, close the Advanced window and click Save changes.

Changing the settings through annotations¶

The data browsing settings can also be set using MDS annotations. The two annotations that allow this are @UIDisplayable and @UIFilterable. Similar to the @Field annotation, they can be placed on fields, as well as on getters and setters. The @UIFilterable annotation will work only, when placed on the field of a supported type.

By default, all fields defined in the entity will be marked as displayable. The @UIDisplayable annotation allows changing this behaviour. If at least one field is marked with the @UIDisplayable annotation, the default behaviour will not be applied, and only annotated fields will be marked displayable. The annotation contains optional position parameter, that allows to pick the position of the field on the data browser UI. The ordering should start with the number zero. Fields are not UIFilterable by default. To allow filtering by field values on the data browser, simply annotate that field with @UIFilterable.

The following code presents the usage of the two annotations:

@Field
private String externalId;

@Field
@UIDisplayable(position = 0)
private String name;

@Field
@UIDisplayable(position = 2)
@UIFilterable
private DateTime dateTime;

@Field
@UIDisplayable(position = 3)
private Long priority;

@Field
@UIDisplayable(position = 1)
private String description;

Changing the UI representation of relationship type fields through annotation¶

The way relationship type fields are displayed can be changed through the @UIRepresentation annotation. This annotation can be placed on a method which takes no arguments and returns String. The @UIRepresentation annotation works only when placed on supported method.

By default, the toString method of an entity would be used to get the display value. You can customize this with the @UIRepresentation annotation.

The following code presents the usage of the annotation

@Field
private String externalId;

@Field
private String name;

@UIRepresentation
public String displayValue() {
        return "Sample Display Value";
}

REST endpoints¶

The general endpoint to the MDS REST operations is: http://<motech-server-address>/module/mds/rest/<<path>>

The table below explains what HTTP request method are supported for each of the CRUD operation, as well as how the “path” should look like.

Response codes¶

These are the response codes returned by the MDS REST API:

• 200 OK - The operation was successful. Note that delete is idempotent, meaning 200 will be also returned for already deleted items.
• 400 Bad Request - The body or parameters provided in the request are invalid.
• 401 Unauthorized - The caller is not authorized and thus not permitted to execute the operation.
• 403 Forbidden - The user does not have necessary rights to execute the operation.
• 404 Not Found - Either the given entity or the requested object does not exist.
• 500 Internal Server Error - The request cannot be processed due to a server error.

Read response¶

In case of read operations Motech also adds metadata to the response. Response is divided into two sections: metadata and data. The metadata contains following fields:

Below you can find sample response:

{
  "metadata": {
    "entity": "EmailRecord",
    "className": "org.motechproject.email.domain.EmailRecord",
    "module": "MOTECH Platform Email",
    "namespace": "",
    "totalCount": 2,
    "page": 1,
    "pageSize": 20
  },
  "data": [
    {
      "id": 1,
      "creator": "admin",
      "owner": "admin",
      "modifiedBy": "admin",
      "deliveryStatus": "SENT",
      "toAddress": "adress1@organisation.com",
      "subject": "Subject 1",
      "message": "Sample message",
      "fromAddress": "adress2@organisation.com",
    },
    {
      "id": 2,
      "creator": "admin",
      "owner": "admin",
      "modifiedBy": "admin",
      "deliveryStatus": "SENT",
      "toAddress": "adress1@organisation.com",
      "subject": "Subject 2",
      "message": "Other message",
      "fromAddress": "adress2@organisation.com",
    }
  ]
}

Parameters and lookups¶

When retrieving the instances using MDS REST API (GET request), there’s an ability to apply some parameters, to have a better control on the result of the request. The parameters are applied as any other GET request parameters.

• id Return a single instance, with the provided id
• pageSize Defines an amount of instances that should be returned per request (defaults to 20)
• page Defines a result page that should be returned (defaults to 1)
• sort Defines a column that should be used to sort the instances in the result
• order Either “asc” or “desc”
• lookup A name of lookup that should be used to retrieve the instances. A lookup must be marked as exposed via REST in order for this to work. The values used in the lookup should be provided as GET request parameters. This an alternative way of calling a lookup, rather than calling it through the lookup url described above.

Below, you will find some examples of valid REST URLs. Assume our entity is called MyEntity.

• http://<<address>>:<<port>>/motech-platform-server/module/mds/rest/MyEntity Return 20 records from the first page (default settings applied)
• http://<<address>>:<<port>>/motech-platform-server/module/mds/rest/MyEntity?id=15 Return an instance with id 15
• http://<<address>>:<<port>>/motech-platform-server/module/mds/rest/MyEntity?page=2&pageSize=50&sort=name&order=asc Return 50 records from the second page, having sorted the instances by name field ascending
• http://<<address>>:<<port>>/motech-platform-server/module/mds/rest/MyEntity?lookup=byName&name=Laura Executes a lookup named “byName” with the lookup field “name” being “Laura” on the entity “MyEntity” and returns results.

REST fields exposed¶

By default all fields are marked as exposed via REST, both for DDE and EUDE. If you choose to hide some of them, they will simply be ignored, when performing CRUD operations via REST on them. When retrieving instances, the result will not contain the fields that are not exposed and when updating or creating instances, the hidden fields will be ignored, even if they are present in the provided JSON representation.

Changing REST settings through the UI¶

You can access the REST API settings by selecting an entity in the Schema Editor and then opening the advanced settings, by clicking on the Advanced button. On the new window, navigate to the REST API tab.

The settings may contain up to three sections:

• The first one, named Fields allows to pick fields that should be exposed via REST. Fields in the table to the right are exposed and fields in the table to the left are not. You can drag and drop fields from one table to another or select them and use provided buttons.
• The next section is named Actions and defines the operations on the instances that are allowed via REST for this entity. By default, no action is allowed. You can choose to change it, by selecting some or all of the actions.
• The last section, called Lookups will appear only if there is at least one lookup defined for an entity. This section allows to pick the lookups that can be executed via REST. Note, that to execute lookups at all, a “Read” action must be enabled.

Changing REST settings through annotations¶

The REST settings can also be applied using MDS annotations. The three annotations that allow this, are:

• @org.motechproject.mds.annotations.RestIgnore As stated in the previous sections, be default all fields are exposed via REST. You can adjust this behaviour using this annotation. Annotated fields will not be exposed via REST.
• @org.motechproject.mds.annotations.RestOperations Placed on the entity class definition, specifies the REST operations that should be allowed for this entity. The annotation takes an array of org.motechproject.mds.annotations.RestOperation, which is an enum of possible values.
• @org.motechproject.mds.annotations.RestExposed Placed on the lookup method definition, in the service interface. Annotated lookup methods will be marked as exposed via REST. By default, lookups are not exposed via REST.

The code below shows an example usage of the annotations:

@Entity
@RestOperations({RestOperation.CREATE, RestOperation.READ})
public class MyEntity {

    @Field
    @RestIgnore
    private Integer number;

    @Field
    private String emailAddress;

    @Field
    private String message;
}


public interface MyEntityService extends MotechDataService<MyEntity> {

    @Lookup(name = "By number")
    List<MyEntity> findByNumber(@LookupField(name = "number") Integer number);

    @Lookup(name = "By Email Address")
    @RestExposed
    List<MyEntity> findByEmailNumber(@LookupField(name = "emailAddress") String emailAddress);
}

REST documentation¶

MOTECH provides a user interface that documents and allows the testing of the REST API exposed by MDS. This interface is generated using Swagger. In order to access this UI, first select REST API in the top menu, then Data Services in the sub-menu.

The raw Swagger specification file (JSON format) is accessible at <your_motech_url>/module/mds/rest-doc.

Configuring validations through the UI¶

To set up validations for a field of an entity, open the Schema Editor and select an entity, for which you wish to set validations. Expand the field that should be validated and navigate to the Validation tab.

Only some of the MDS types support setting up validations via UI, so if a selected field is of a type that is not supported, the Validation tab will not appear. Please see the list of supported types and validations below.

The Regex validation contains some predefined patterns, for the most common use cases. To view them, click Select, next to the Regex input field and pick one of the available, predefined expression. This will automatically, place the regular expression in the input field. Please note, that this operation will erase the current value in the field, if there’s any provided.

Setting up validations will display hints while adding an instance of an entity, that has got validated fields. An attempt to add an instance with invalid values, will display an error and block the ability to save the instance.

Configuring validations using annotations¶

For DDEs, it is possible to set up validations using the annotations. MDS will recognize the @javax.validation.constraints annotations, as well as two MDS-defined annotations: @org.motechproject.mds.annotations.InSet and @org.motechproject.mds.annotations.NotInSet. See the code below, for an example of validation definition through annotations.

@Entity
public class MyEntity {

    @Field
    @Min(10)
    @Max(100)
    private Integer number;

    @Field
    @Pattern(regexp = "^\\w+([\\.-]?\\w+)*@\\w+([\\.-]?\\w+)*(\\.\\w{2,3})+$")
    private String emailAddress;

    @Field
    @AssertTrue
    private Boolean alwaysTrue;

    @Field
    @Size(min = 64, max = 2048)
    private String message;
}

Even though you can use any @javax.validation.constraints annotation on an entity field, the UI support (hints, error messages), will only be displayed for the validations listed in the previous section, about setting validation through UI. Other validations will not show up on the UI, but it still will not be possible to add an invalid value - a ConstraintViolationException will be thrown.

Executing JDO queries¶

MDS allows developers to use the JDO API offered by DataNucleus to execute any query they wish. A utility method for calling direct SQL queries through DataNucleus. Although the approach of executing custom queries gives the user all the flexibility he needs, the more easier and recommended approach is to use Lookups instead. This API remains in place however in order to fulfil the more complex requirements.

In order the execute a custom JDO query, the developer has to implement the org.motechproject.mds.query.QueryExecution interface and pass an instance of this implementation to the executeQuery(QueryExecution) method. This interface exposes one method - execute(javax.jdo.Query, org.motechproject.mds.util.InstanceSecurityRestriction). The first a parameter is the javax.jdo.Query instance class created using the PersistenceManager for the entity class of the data service being used, the second is an object describing security restrictions on the entity.

What is returned by the interface method will be also returned by the executeQuery() call on the data service. The interface is generic, the type parameter represents the return value.

Following is an example of executing a custom JDO query. Given a simple entity:

@Entity
public class Example {

    public Integer amount;

    public String name;
}

Here is an example of a JDO query that will check the amount value and based on that select only the names from the database:

// get the service for the entity you wish to execute the query on
MotechDataService<Example> service = getService();

QueryExecution<List<String>> queryExecution = new QueryExecution<List<String>>() {
    @Override
    public List<String> execute(Query query, InstanceSecurityRestriction restriction) {
        // return objects with the amount value either less then 1000 or greater then 1000
        query.setFilter("amount < 100 || amount > 1000");

        // select only the name column
        query.setResult("name");

        // limit the results
        query.setRange(0, 100);

        return (List<String>) query.execute();
    }
};

List<String> names = service.executeQuery(queryExecution);

More info on JDO queries can be found here: http://www.datanucleus.org/products/datanucleus/jdo/jdoql.html

Executing SQL queries¶

Similar to executing JDO queries MDS also provides developers with access to executing SQL queries. Instead of implementing the QueryExecution interface however, developers have to implement the org.motechproject.mds.query.SqlQueryExecution interface. This interface has two methods, execute(javax.jdo.Query) and getSqlQuery(). The contents of the SQL query should be returned by the getSqlQuery methods, so that MDS can construct the JDO query using that SQL.

Following is an example of executing a custom SQL query. Given a simple entity:

@Entity
public class Example {

    public Integer amount;

    public String name;
}

Here is an example of a SQL query that will return values with the given amount:

// there is really no impact on which data service is used, since this is raw sql
MotechDataService<Example> service = getService();

SqlQueryExecution<List<String>> sqlQueryExecution = new SqlQueryExecution<List<String>>() {
    @Override
    public List<String> execute(Query query) {
        // usage of params
        Map<String, Integer> params = new HashMap<>();
        params.put("param", 5);
        return (List<String>) query.executeWithMap(params);
    }

    @Override
    public String getSqlQuery() {
        // this query will be executed by MDS
        return "SELECT name FROM MDS_EXAMPLE WHERE amount = :param";
    }
};

List<String> names = service.executeSQLQuery(sqlQueryExecution);

Note that using raw SQL should be the absolute last resort, it is advised to stick to more high-level concepts in your code.

Access to the Data Services module¶

MDS registers three permissions, that restrict access to certain parts of the Data Services module via MOTECH UI. They are:

• mdsSchemaAccess (grants access to the Schema Editor)
• mdsDataAccess (grants access to the Data Browser)
• mdsSettingsAccess (grants access to the Settings panel)

The MDS Admin role contains all of these three permissions.

Access to the instances¶

Depending on the chosen option, two security levels can be recognised in MDS:

Security settings can be set through the UI or by the @org.motechproject.mds.annotations.Access annotation for DDE. It works only with the @Entity annotation.

There are five security modes:

The code below shows an example usage of the annotation:

@Entity
@Access(value = SecurityMode.ROLES, members = {"admin"})
public class MyEntity { }

To update security settings via UI, pick the entity and click the Security button.

A new modal window will appear, where security settings can be updated.

Tasks integration¶

For the entities that expose these events, you can create tasks with these events as a trigger. To do it go to the Task module, click ‘New task’ and you should see the Data Services trigger list. A trigger is exposed for every crud event per entity:

In the Task module, you can also use Data Services as a channel and select an action you want :

Account settings¶

To allow MOTECH to connect to the CommCareHQ, you should first add a new configuration and provide correct credentials to the CommCareHQ account:

• CommCare Base URL (a link to the CommCareHQ instance; by default www.commcarehq.org)
• CommCare Domain (project name on CommCareHQ)
• Username
• Password

To add configuration you must use Add configuration button . After filling in the data, press the Save button.

To verify the provided data, click the Verify button. The Commcare module will send a test request to the CommCareHQ, attempting to authenticate with the credentials you have provided. If everything works OK, you will be notified about successful connection. If there were any problems connecting to the CommCareHQ, an error will be displayed and you will not be able to work with the Commcare module, until valid credentials are provided.

You can provide more than one account configuration. Only one of the supplied configuration can be the default. The default configuration will be selected whenever you do not specify a particular configuration. To mark the configuration as default you must save the configuration(if it is new) and use Make default button.

Event forwarding¶

This section allows you to configure the events, fired by the Commcare module. Currently, you can pick the Event Strategy for the forwarding of case events. There are three options available:

• minimal (the event will contain only the case ID)
• partial (the event will contain case ID, as well as other, not-case specific parameters (case metadata)
• full (the event will contain case ID, case metadata and all field values of a case)

Once you switch the option, the events fired by the Commcare module will contain only the fields you have chosen to forward.

Connect CommCareHQ¶

To let CommCareHQ know, where the data should be forwarded, you also need to set up data forwarding URLs. This can be achieved in two ways. The first way is to do it via the checkboxes. If you want to set up CommCareHQ to forward the data to the Commcare module, select the checkbox and the Commcare module will automatically set up an URL on your CommCareHQ account.

If for any reason, the first way doesn’t work or sets up invalid URL, the data forwarding rules can also be set in the project settings on your CommCareHQ account. If you have enabled the rules via the slider buttons, you can also verify that the correct rules (with proper URL to your server) have been set up on the CommCareHQ.

To disable the data forwarding rules, you have to open the project settings on your CommCareHQ account and disable them from there. The Commcare module can only set up the rules, but cannot disable them.

As you can see the provided URL http://demo.motechproject.org/module/commcare/forms/ doesn’t have specified configuration. So if the Commcare module receives data, the default configuration will be used. To work with more than one configuration you will have to use for example such URL http://demo.motecproject.org/module/commcare/forms/myProjectConf. The URL with the configuration name should be automatically added by the Commcare module.

Channels¶

A channel contains information about all exposed triggers and actions within a given module. It can be considered as a module specific configuration that tells the Tasks module how it can make a use of it.

The most important elements of the channel are trigger and action definitions. In fact, if channel does not define neither triggers nor actions it is considered invalid. Other properties includes the channel display name, that will be visible on the Tasks UI (it may be a key from message.properties, in which case it will appear as a translated message). Module version and name are obtained at channel registration from the registering bundle. Additionally the channel may contain a short description.

Detailed definition:

Triggers¶

A trigger represents a precise definition of events exposed by module. In tasks, a trigger is something that, as the name suggests, triggers task executions. This means that when an event described by the trigger is published, all tasks with that trigger will get executed. Every trigger has an unique name and, in a simple case, corresponds to exactly one event. Parameters of this event are defined within the trigger. Each parameter contains its name (key) and type.

A good example of a trigger can be an inbound SMS. It would contain the following parameters: message (STRING), sender (STRING), recipients (LIST) etc. Those information will be accessible in the Tasks module.

In the basic case, the most important elements of a trigger are subject and event parameters. The subject corresponds to the event subject that is wrapped by this trigger, while event parameters are the parameters that will be exposed by the trigger. Providing this basic kind of the trigger makes Tasks module listen to the event with the given subject. Each time such an event is published, all active tasks with a corresponding trigger are executed.

However, in some cases the basic behaviour is not sufficient. Sometimes we want the event to correspond to many triggers. In this situation, the trigger listener subject comes in handy. It has to be used along with a custom event parser, which is a little more advanced component, thus it will be described later.

Detailed trigger definition:

Detailed trigger event parameter definition:

Actions¶

An action represents a definition of function that can be called in a response to a trigger. Every action can represent either a single method of an OSGi service that will be called or an event that will be sent. Each parameter contains it name (key), type and may contain its default value. In case of a method call, the way in which parameters will be passed may vary depending on the needs. They can be either passed directly to the method (matching its signature) or using a key-value pair map.

For instance, an action may correspond to sending an email message. That action would then contain some required fields such as recipients (as a LIST) and the message (STRING) and some optional fields, for example the delivery time (DATE).

As mentioned before, there are two forms in which an action can be represented. The first one is an event. In this case, the action must define a subject of that event. Action execution leads to creating an event with the defined subject and parameters that correspond to the exposed action parameters. The second form that action can take is a service method call. In that case, the action definition must contain the name of the OSGi exposed service interface and the method name to execute. Additionally, one can specify the way in which the method will be called. When it is specified as ‘named parameters’, the action parameters will be evaluated, casted and passed directly to the service method according to its signature and matching its parameter names. In the other case, when it is specified as ‘map’, the parameters are evaluated, packed into a hash map and passed to the method. In this situation the service method is supposed to take exactly one parameter of type java.util.Map<java.lang.String, java.lang.Object>.

An action is considered invalid if it does not define the method nor the event. However, it can define both of them, but the method call has the precedence before event passing. Thus event is send only if the method defining service is not available.

Detailed action definition:

Detailed action parameter definition:

Parameters types¶

Available types that can be used with action parameters and trigger event parameters in the Tasks module are listed below.

Data providers¶

A data provider can be considered as a source of various data that can be used in a task. It defines the structure of objects it supports as well as structure of the queries that it can perform. Each data provider is recognized by its name.

An example data provider is the one defined by the CMSLite module. It provides two types of objects: StreamContent and StringContent. For instance, StringContent objects contains several fields that can be used in the Tasks module. Those are value, language, name and metadata. It also contains two lookups. One of them is used to find a desired instance by id, the other one uses name and language fields.

Every data provider must implement the DataProvider interface. It contains a few methods responsible for retrieving the data provider name, performing a search, discriminating if a provided type is supported by this provider and finally, returning the provider JSON definition. The definition is in fact a TaskDataProvider object, thus it must follow its schema.

Detailed task data provider definition:

Detailed task data provider object definition:

Detailed lookup field parameter definition:

Detailed field parameter definition:

Using the channel file¶

It is the most common way to register a task channel. It comes down to creating a json channel definition file named task-channel.json and placing it right in the classpath root of your bundle. It will be automatically discovered by the Tasks module at your bundle start or update.

The file content, written in JSON format, has to follow a well defined structure. The root element must be a channel object that matches a specification defined above.

Example channel file:

{
    "displayName": "sms",
    "triggerTaskEvents": [
        {
            "displayName": "sms.inbound_sms",
            "subject": "inbound_sms",
            "serviceInterface": "org.project.service.SmsService",
            "serviceMethod": "sendSms",
            "eventParameters": [
                {
                    "displayName": "sms.message",
                    "eventKey": "message"
                },
                {
                    "displayName": "sms.sender",
                    "eventKey": "sender"
                },
                {
                    "displayName": "sms.recipient",
                    "eventKey": "recipient"
                },
                {
                    "displayName": "sms.datetime",
                    "eventKey": "datetime",
                    "type": "DATE"
                }
            ]
        }
    ],
    "actionTaskEvents": [
        {
            "displayName" : "sms.send_sms",
            "subject" : "send_sms",
            "actionParameters" : [
                {
                    "displayName" : "sms.message",
                    "key" : "message"
                },
                {
                    "displayName" : "sms.recipients",
                    "key" : "recipients",
                    "type" : "LIST"
                },
                {
                    "displayName" : "sms.delivery_time",
                    "key" : "delivery_time",
                    "type" : "DATE",
                    "required": false
                }
            ]
        }
    ]
}

Using annotations¶

This method allows to register a channel using the Tasks annotation processing mechanism and annotations from org.motechproject.tasks.annotations package. However, this approach is limited to registering actions only. In this scenario, channels correspond to classes and actions to their methods. To make a class recognized as a channel by the Tasks module, it has to be annotated with @TaskChannel. Channel display name can be provided as an annotation parameter. Additionally, module name and version have to be provided as annotation parameters.

Each channel class should have at least one method marked as @TaskAction. For this annotation as well, one can specify the action display name. Each parameter of the action method is considered as an action parameter with default properties: the parameter are marked as required, its type is set to UNICODE and its display name and key corresponds to the action method parameter name. Those default properties can be modified utilising the @TaskActionParam annotation.

Example channel class:

@Service
@TaskChannel(channelName = "sms", moduleName = "sms", moduleVersion = "1.0")
public class SmsServiceImpl implements SmsService {

    @TaskAction
    public void sendSms(
        @TaskActionParam(displayName = "sms.message", key = "message") String message,
        @TaskActionParam(displayName = "sms.recipients", key = "recipients", type = ParameterType.LIST) List recipients,
        @TaskActionParam(displayName = "sms.delivery_time", key = "delivery_time", type = ParameterType.DATE) DateTime deliveryTime
        ) {

        ...
    }

}

Using the ChannelService¶

The most elastic way to register a channel is to use the ChannelService. It allows to register both triggers and actions in a dynamic manner. The first step of typical usage of the ChannelService is to build a ChannelRequest object. The ChannelRequest is the Java representation of a channel, that follows already defined channel specification. Accordingly, TriggerEventRequest and EventParameterRequest corresponds to trigger and trigger event parameters and ActionEventRequest and ActionParameterRequest corresponds to action and action parameters. Note that in this scenario module name and module version must be provided manually as proper fields of the request. Once the ChannelRequest is ready, it can be passed to the ChannelService method called registerChannel. It will validate the request and register the tasks channel.

Example channel registration using the ChannelService:

@Component
public class SmsChannelRegistration {

    @Autowired
    private ChannelService channelService;

    ...

    private void registerSmsChannel() {

        EventParameterRequest inboundSmsMessage = new EventParameterRequest(
            "message", // event key
            "sms.message" // display name
        );

        EventParameterRequest inboundSmsSender = new EventParameterRequest(
            "sender", // event key
            "sms.sender" // display name
        );

        EventParameterRequest inboundSmsRecipient = new EventParameterRequest(
            "recipient", // event key
            "sms.recipient" // display name
        );

        EventParameterRequest inboundSmsDatetime = new EventParameterRequest(
            "datetime", // event key
            "sms.datetime", // display name
            "DATE" // type
        );

        TriggerEventRequest inboundSmsTrigger = new TriggerEventRequest(
            "sms.inbound_sms", // display name
            "inbound_sms", // subject
            null, // description
            Arrays.asList(inboundSmsMessage, inboundSmsSender, inboundSmsRecipient, inboundSmsDatetime) // event parameters
        );

        ActionParameterRequest sendSmsMessage = new ActionParameterRequest(
            "message", // key
            null, // default value
            "sms.message", // display name
            0, // order
            null, // type (default: UNICODE)
            true, // required
            false // hidden
        );

        ActionParameterRequest sendSms = new ActionParameterRequest(
            "recipients", // key
            null, // default value
            "sms.recipients", // display name
            1, // order
            "LIST", // type (default: UNICODE)
            true, // required
            false // hidden
        );

        ActionParameterRequest sendSms = new ActionParameterRequest(
            "delivery_time", // key
            null, // default value
            "sms.delivery_time", // display name
            2, // order
            "DATE", // type (default: UNICODE)
            false, // required
            false // hidden
        );

        ActionEventRequest sendSmsAction = new ActionEventRequest(
            null, // name
            "sms.send_sms", // display name
            "send_sms", // subject
            null, // description
            "org.project.service.SmsService", // service interface
            "sendSms", // service method
            null, // service method call manner (default: NAMED_PARAMETERS)
            Arrays.asList(sendSmsMessage, sendSmsRecipients, sendSmsDeliveryTime) // action parameters
        );

        ChannelRequest smsChannel = new ChannelRequest(
            "sms", // display name
            "sms", // module name
            "1.0", // module version
            null, // description
            Arrays.asList(inboundSmsTrigger), // trigger requests
            Arrays.asList(sendSmsAction) // action requests
        );

        channelService.registerChannel(smsChannel);
    }
}

Overview¶

The main Tasks view contains a few elements. Firstly, the action buttons are on the top. They allow creating tasks, importing previously exported tasks and toggling the visibility of the filter view on the right.

The main view lists all currently existing tasks in the form of expandable boxes, that provide actions related to the tasks they represent. The list can be filtered using filters tab mentioned before. One can search the tasks by their name or description as well as be their state (active/paused).

The right panel, besides filters, contains also a recent task activity tab. It provides an instant overview of latest task executions and their results.

Creating a task¶

New task creation process begins with clicking the ‘New task’ button on the main view. The task creation view shows up.

Starting from the top, one can see two properties to provide: task name and task description, from which the task name is mandatory.

The trigger selection widget comes next. In this example there are four channels registered that expose at least one trigger. Once the channel icon is clicked, a popup shows up. It lists all triggers exposed by that channel.

Picking a trigger makes new actions available. One can add a data source, a filter set and finally select an action to execute.

After clicking the ‘Add data source’ button, the data source widget shows up. The first step is to select an actual source. The dropdown lists all registered data sources. After picking one, a data source object must be selected. Now, one has to choose a lookup that will be used to retrieve an object and provide its arguments. In this example the ‘find by id’ lookup is used, thus the only lookup parameter is ID. The argument value may be either entered by hand (hardcoded) or composed from available fields listed on the top of the widget. Additionally, it is possible to set that task execution will fail if the lookup will not find the desired object.

Another available option is to add a filter set. The filter allows to setup a set of conditions that must evaluate to true in order for the task to execute. One can choose if all conditions should be satisfied or just one of them. If the entire condition set is not fulfilled, the task execution is canceled.

Each filter corresponds to a single field, either from trigger or data source. After selecting a field, there is a possibility to manipulate its value using Tasks manipulations by clicking on the gear icon next to it. Then it has to be set if the filter should be satisfied on a condition result or its negation. Finally, the condition can be selected. All conditions are grouped in three categories including Date, Number and String. Each of those contains basic checks that can be performed on the types they represents.

The filter is executed after all previous steps (data source lookups, filters) before were executed. In order for a filter to perform a check on a given data provider object, it has to be placed after that data source step. This can be used to abort the task execution before doing costly data lookups.

At last, with the ‘Add action’ button there comes a possibility to add an action to a task. There can be multiple actions in a single task, but unlike filters and data sources, the task is obligated to contain at least one action to be valid.

All channels that expose at least one action are listed in the channel dropdown. When one of them is selected, the action dropdown contains all actions available. After selecting both channel and action, a list of action parameters is presented. In order to be valid, the action must not contain any parameter with an invalid value. Like in the case of data source lookups, the parameters may be filled with hardcoded values or combined field available either from the trigger or a data source. In case of fields, its values can be modified using Tasks manipulations. Sometimes, when a property has a complex type, a question mark can be visible next to its label. When hovered over, a popup with a short tooltip is shown.

There are also two buttons at the bottom on the action widget. Once clicked, they provide a handy user manual related with fields syntax and string/date manipulations.

Once the task is defined, the last thing to do is to save it. There are two buttons on the bottom that allow to achieve this goal. One of them simply saves the task. This action is possible even if the task is not fully valid. The second option is to save and enable the task at once. In this case, the task must be valid. After clicking any of the buttons, the saved task can be seen in the main view.

Manipulations¶

In various situations related with task creation, there is a possibility to apply so called manipulations to fields originating from trigger or data sources. Manipulation allow to modify the incoming field values and transform them to something else. The most basic example might be changing all letters of a string value to uppercase.

Depending on the field type, distinct manipulations may be enabled. There are currently two categories of supported types: String and Date. An extensive description of them is available at the Tasks UI through a proper help button.

There might be multiple manipulations assigned to a single field. Moreover, they can be ordered in the widget by simply dragging and dropping them.

There is also possibility to define manipulations ‘by hand’ using plain text. The syntax in this case is {{[field]?[manipulation]}}, for example {{trigger.externalId?substring(0,8)?toUpper}}.

Managing and monitoring tasks¶

Once the task is created, it is shown in the main Tasks view in a widget form. In a basic form it contains information about modules related with the task, the task name, an icon indicating if the task is active or not and an icon that leads to a task editing view. Once the task name is clicked, the widget expands to expose all available actions related that task.

The first action allows to edit the task. The editin process is very similar to task creation. The editor view shows up and presents the task in a form in which it was saved. In a situation when a task is invalid, all validation errors are visible at the top of the view.

Second button toggles the task state between paused and active. Active task will be executed when their corresponding trigger will occur, while paused task will not. The task may be paused in order to temporarily disable the task execution.

The delete button allows to permanently delete tasks. Once deleted, a task cannot be restored.

There is also a button that leads to the task history view. It allows to monitor all events related to the task and especially track an execution of the task. It provides information about the task result status and the message, which in case of failure contains stacktrace and failure reason. You can also clean the tasks history.

The last available option is to export the task. Selecting this action will trigger a file download. The file is a json representation of the task, that can be imported using ‘Import task’ action in the main view.

Settings¶

The only setting actually available is the limit of invalid executions for a single tasks. If the task will fails more times than it is allowed to by this parameter, it will be automatically paused until its manual activation. Once this happens, a message is added to the task events history. If the value of this parameter is set to 0, the task will be paused after only one failure.

It is worth mentioning that this parameter may be also set using file based config. The property name of the parameter is ‘task.possible.errors’.

Stateless¶

A core design principle of the MOTECH platform is that the server should be stateless across requests to allow for horizontal scalability. It is expected that code running within the MOTECH server should perform a single action per request and then return. The module should never persist any state in memory or local disk and expect that state to be available to later requests.

Events¶

To aid in the development of stateless services, the MOTECH engine provides a pub/sub like event system. (The event system follows the publish-subscribe pattern but does not implement the standard Java pub/sub protocol.) It helps to decouple emitters of events from the modules that wish to consume them. Any module can emit an event by calling the EventRelay and passing it a MotechEvent and a subject. To register for an event, a module just needs to annotate a method with the list of event subjects of interest.

For more information, see Event and Scheduler Architecture.

Scheduled Events & Timers¶

To assist in the development of a stateless event-based server, the MOTECH platform provides access to a flexible scheduling system. Using the open source Quartz engine, MOTECH can easily schedule events for future consumption. For more information, see Event and Scheduler Architecture.

Tasks System¶

The Tasks system allows you to connect modules without code by using tasks. Each task consists of three parts:

1. Trigger: an event raised by Module A (or the Scheduler)
2. Filter: a conditional statement specifying whether the task should run
3. Action: an action executed by Module B in response

In between the trigger and the action, tasks may use data loaders to look up data from other modules that are registered as data sources.

Data Services¶

MOTECH Data Services is a flexible data modeling system that allows users to define and share custom schemas without code, and provides auditing and revision tracking. It is a JDBC-based user configurable database abstraction layer on top of a standard SQL database. It provides generated POJOs and OSGi service interfaces for the data objects, generated CRUD events, and generated user interface for data browsing and editing. In a future release it will also support auto-generation of REST APIs.

Quartz Scheduler¶

Quartz is an open source job scheduling engine that enables MOTECH modules to schedule events for future consumption.

Tomcat¶

Apache Tomcat provides the application container for MOTECH.

ActiveMQ¶

Apache ActiveMQ is an open source message broker that provides the message queue and the message topic.

OSGi¶

Each MOTECH module is an OSGi bundle. Using OSGi allows the platform to manage the bundle lifecycle (adding, removing, starting, and stopping modules), and allows modules to expose service interfaces. For more information, see Modules Architecture.

Notes¶

• The scheduler service retrieves the Quartz Scheduler from the SchedulerFactoryBean. The service can only schedule MotechScheduledJobs, which all must include a MotechEvent.
• When the trigger for the scheduled job is satisfied, the job is executed and the accompanying Motech event is sent to the SchedulerFireEventGateway interface. This gateway is defined in schedulerFiredEventChannelAdapter.xml and a proxy is generated by Spring at runtime.
• The SchedulerFireEventGateway serves as an outbound message gateway to shuttle messages to the event relay, which then sends JMS messages to the event queue.
• The event queue dequeues messages and routes them to the event listener registry. The core platform has one consumer for events, a channel that routes messages to an event listener registry.
• The event listener registry retrieves the listeners that are listening on the motech event it is relaying (listening is based on the value bound to the motech event’s subject key).
• The core platform scans new bundles for any method annotated as a MotechListener, and registers those methods with the listener registry.
• If the event has a specific destination or only one listener, the listener’s handle method is called. This will invoke the method that was annotated as a MotechListener. These listener classes can be project specific and will reside outside of the core platform.
• If the event has multiple listeners, a separate event is raised for each listener. The original event object is not handled by a listener in this case.
• The EventListener’s handle method will invoke the method that was annotated as a MotechListener. The MotechEvent will be passed to that method as a parameter.
• If you wish to use ActiveMQ web console to view MotechEvents, please note that the server running the console must have the motech-platform-event bundle in its classpath. Therefore, either place the jar in the default location, or add a classpath that will contain the motech-platform-event jar.
• Note that MOTECH Scheduler is not a real-time system. There’s no guarantee that your scheduled job will fire the exact same minute it has been scheduled for. Everything depends on the present CPU usage, other jobs that must be served and many more factors. Pay special attention to this, when scheduling jobs close to midnight, in case the date matters in your use case.

Default MOTECH roles¶

MOTECH defines 5 default roles.

Description¶

To create your own password validator you must create new validator class which will implement PasswordValidator interface. The next step is to create OSGI service from your new validator. To do this you must add @Service annotation to the class and service reference information in the blueprint file.

PasswordValidator interface contains following methods:

• void validate(String password) throws PasswordValidatorException - validates password.
• String getValidationError(Locale locale) - returns the error message(should be treated as a literal) for the validator. Message should explain what is expected in password.
• String getName() - Returns the name of the validator used for retrieval. Must match the value from the configuration in order to be used.

Example Code¶

Below you can find example of a validator which requires 3 special characters in password.

@Service("serviceName")
public class MyValidator implements PasswordValidator {

    private String name = "validator_name";

    @Override
    public void validate(String password) {
        CharacterCount count = new CharacterCount(password);

        if (count.getSpecial() < 3) {
            throw new PasswordValidatorException("Invalid password, validator name - " + name);
        }
    }

    @Override
    public String getValidationError(Locale locale) {
        return "Password must have 3 special characters";
    }

    @Override
    public String getName() {
        return name;
    }
}

To enable it you must add <osgi:service ref="serviceName" interface="org.motechproject.security.validator.PasswordValidator"/> to the blueprint file and set it in config(security.password.validator=validator_name).