How to build a cloud connector for Mule ESB

I'm working on a particular project which uses Mule ESB as a central piece, for adding a new feature to my project I need to consume Google's custom search API so for making it beautiful I'm going to create a connector using Mule's DevKit and showing step by step the process of building a connector that consumes a REST API.

First, I create a DevKit module from the archetype:

mvn archetype:generate -DarchetypeGroupId=org.mule.tools.devkit -DarchetypeArtifactId=mule-devkit-archetype-generic -DarchetypeVersion=3.4.0 -DarchetypeRepository=http://repository.mulesoft.org/releases/ -DgroupId=com.mycompany -DartifactId=mule-module-google-custom-search -Dversion=1.0.0-SNAPSHOT -DmuleVersion=3.4.0 -DmuleModuleName=GoogleSearch -Dpackage=com.mulesoft.module.googlesearch -DarchetypeRepository=http://repository.mulesoft.org/releases

Since I'm building a cloud connector for google's custom search API it is handy to have it's reference at hand:

https://developers.google.com/custom-search/v1/cse/list 

You may also want to create a custom search engine:

http://www.google.com/cse/create/new

Lastly: here is the link for the source code of the project:

https://github.com/juancavallotti/mule-module-google-custom-search

Now let's the coding begin! First of all I want to clean up the sample module, pretty much removing everything except the class and it's annotations. I have picked to be a module (instead of a proper connector) because it's a fair simple functionality which does not require work for setting up a connection. The class looks like this:

1 2 3 4 5 6 7 8 9

/** * Google custom search Module. * * @author Juan Alberto López Cavallotti. */ @Module(name="google-search", schemaVersion="1.0.0-SNAPSHOT") public class GoogleSearchModule { }

So first we will add the required configuration parameters as stated at the documentation, these parameters are the API Key, and the custom search engine ID or it's url (one of both are required). The search query is also required but we'll save it for passing it as a parameter to the only operation this connector will have. I will give nice names to these parameters instead of the really short ones they currently have, this way is more self-documented.

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16

@Module(name="google-search", schemaVersion="1.0.0-SNAPSHOT") public class GoogleSearchModule { @Configurable private String apiKey; @Configurable @Optional @Default("") private String searchEngineId; @Configurable @Optional @Default("") private String searchEngineUrl; //getters - setters }

DevKit demands us to write javadoc for each element on the class so it is able to generate the connector's documentation properly. I won't show this documentation on this post unless I have something to say about it, having said that, you might want to take a look at the final version of this module in order to get more insight on its development.

Next, I want to create the one and only operation as a method of this module definition, this method has a lot of parameters, most of all with default values. For me this is pretty ugly and DevKit allows me to do better so I will do, I will pick the most important parameters for direct setting and the others would be gathered through a POJO. The return value I will keep it as the JSON String the service returns (for the sake of simplicity) but kind of more beautiful would be to create an Object Structure and take advantage that Mule has bundled the Jackson Library.

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17

/** * Perform a Google custom search. * * {@sample.xml ../../../doc/GoogleSearchModule-connector.xml.sample google-search:search} * * @param query The search query to send to google search. * @param siteSearch The site where to search. * @param searchType The type of the search to be performed. * @param searchConfiguration Configuration for this google search. * * @return The JSON result as returned by the google custom search API. */ @Processor public String search(String query, @Optional @Default("") String siteSearch, @Optional SearchType searchType, @Optional SearchConfiguration searchConfiguration) { return null; }

You need to consider the following:

• The method must be annotated with one of the annotations provided by devkit to generate different types of message processors. (Each type has its own signature requirements).
• Every operation should be documented properly.
• You need to create a sample usage of the operation on the sample file, (which luckily it gets created by the archetype).
• You need to document EVERY parameter if you wish the build to succeed.

<!-- BEGIN_INCLUDE(google-search:search) -->
<google-search:search query="#[header:inbound:query]" />
<!-- END_INCLUDE(google-search:search) -->

Now, the sample XML looks like the following:


I want to perform an initial configuration and validation when the module starts just to make sure we're ready to go when making a search so I create a method to take advantage of the configuration element lifecycle:

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22

private HashMap<String, String> connectorConfigs; /** * Perform module initialization. */ @Start public void initializeConfiguration() { connectorConfigs = new HashMap<String, String>(); connectorConfigs.put("key", apiKey); if (StringUtils.isBlank(searchEngineId) && StringUtils.isBlank(searchEngineUrl)) { throw new IllegalArgumentException("You must configure either searchEngineId or searchEngineUrl"); } if (StringUtils.isNotBlank(searchEngineId) && StringUtils.isNotBlank(searchEngineUrl)) { throw new IllegalArgumentException("You must configure a reference to the custom search engine."); } addIfNotBlank(connectorConfigs, "cx" , searchEngineId); addIfNotBlank(connectorConfigs, "cref", searchEngineUrl); }

This basically creates a map that will be the base parameters for every request. Every request should have at least a reference to the API key (which is the way google charges us for our searches) and the reference to our custom search engine (which we need to create manually in order to use the API).  


Next, let's dig into the actual implementation. I will want to access the MuleEvent for this message processor so, I will modify slightly the signature of the method. What this implementation does is: create a map with the parameters to send to the API, then convert these maps into the API url and finally dispatch it through Mule's HTTPS connector. This is very convenient since it allows the configuration of the HTTPS connector parameters even though there is still room for improvement!! I could have made the http connector to use configurable but I won't just to keep it simple. The search method now looks like this:

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28

@Processor @Inject @Mime("application/json") public String search(MuleEvent event, String query, @Optional @Default("") String siteSearch, @Optional @Default("WEB_SEARCH") SearchType searchType, @Optional SearchConfiguration searchConfiguration) { MuleContext context = event.getMuleContext(); MuleMessage message = event.getMessage(); HashMap<String, String> searchParams = buildSearchParams(query, siteSearch, searchType, searchConfiguration); String apiUrl = buildSearchUrl(searchParams); try { OutboundEndpoint endpoint = context.getEndpointFactory().getOutboundEndpoint(apiUrl); //configure the message. message.setOutboundProperty("http.method", "GET"); MuleEvent responseEvent = endpoint.process(event); //return the payload. return responseEvent.getMessage().getPayload(String.class); } catch (MuleException e) { logger.error("Error while querying the google custom search API", e); } return null; }

This is pretty straightforward. I won't get into the implementation of the auxiliary methods because it is just boilerplate. I want to highlight the @Inject annotation I used, this is used to get the actual MuleEvent (and distinguish this parameter from what the user needs to provide) and also the @Mime annotation which will generate the appropriate message header.

Now we just want to create the MuleStudio update site so we can install and try this module:

mvn clean package -Ddevkit.studio.package.skip=false

We can import this through update site and use it! There are furhter configurations I can (and probably will) make to this connector, DevKit has annotations for customizing the studio dialogs and much more!!

Here is a sample project which uses the recently created connector:

<mule xmlns:http="http://www.mulesoft.org/schema/mule/http" xmlns:google-search="http://www.mulesoft.org/schema/mule/google-search" xmlns="http://www.mulesoft.org/schema/mule/core" xmlns:doc="http://www.mulesoft.org/schema/mule/documentation" xmlns:spring="http://www.springframework.org/schema/beans" version="EE-3.4.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-current.xsd
http://www.mulesoft.org/schema/mule/core http://www.mulesoft.org/schema/mule/core/current/mule.xsd
http://www.mulesoft.org/schema/mule/http http://www.mulesoft.org/schema/mule/http/current/mule-http.xsd
http://www.mulesoft.org/schema/mule/google-search http://www.mulesoft.org/schema/mule/google-search/1.0.0-SNAPSHOT/mule-google-search.xsd">
    
    <google-search:config name="Google_Search" apiKey="<api key>" searchEngineId="<engine id>" doc:name="Google Search"/>
    
    <flow name="mule-configFlow1" doc:name="mule-configFlow1">
        <http:inbound-endpoint exchange-pattern="request-response" host="localhost" port="8081" doc:name="HTTP"/>
        <google-search:search config-ref="Google_Search" query="http connector" doc:name="Google Search" />
        <logger level="ERROR" message="Payload is: #[payload]" doc:name="Logger"/>
        <logger level="ERROR" doc:name="Logger"/>
    </flow>
</mule>

Please continue reading the  DevKit cookbook for much more info:

http://www.mulesoft.org/documentation/display/current/Cloud+Connector+Devkit+Cookbook