End to End Example - A Mule 3.5.0 App with Domains

Recently, I've added to my gradle plugin for building mule apps the ability to build domain-based (A.K.A. mule shared resources) projects. While the plugin gains maturity and is getting ready for prime time, I've started eating my own dog food with it and as a result I've produced a sample application that uses some features of Mule 3.5.0 Community Edition. Please checkout the instructions on how to run the sample project.

The Use Case

Let's imagine that we have in our premises a database of cooking recipes, but we have the requirement of exposing a REST API so users can check available recipes and post new ones.

This is a simple use case but we have some additional requirements as well as some constraints:

1. The Backend APP needs to have the less down time possible.
2. We need to be able to change the API since it will evolve and gain new features.
3. We need to be able to scale the backend so when thousands of users access it at the same time it does not become extremely unresponsive.
4. Upon an upgrade in the backend it would be desirable that users can still post recipes. 

Implementation

For the implementation of these requirements, I've decided to divide and conquer, this is create two modules:

1. API Module: This is responsible for receiving requests through HTTP, and adapt them in a way the backend can understand and process them. Also is responsible for representing appropriately the information provided by the back-end.
2. Backend Module: This module is responsible for accepting requests, processing them and returning the results.

The following is a diagram of the implemented architecture:

The communication between the modules is leveraged through JMS, initially as the app has only a couple of users, we don't want to have a separate JMS Broker, so we'll leverage an embedded broker, but this is easily upgradable to an external broker without making any changes to the Modules.

Backend Implementation

For implementing the backend (source code) I've leveraged the brand new Database Module. This module allows to easily run parameterized queries. The backend module's main interface are JMS queues, and for the operations that require a response, we'll take advantage of mule's automatic reply when ReplyTo-type headers are present.

API Implementation

For implementing the front end (source code), I've used APIKit, this module allows us to easily expose a REST-like API out of a RAML file, so before starting the implementation I went to the Anypoint Platform website and created a RAML File with the API Editor.

In order to implement the requirement of accepting recipes even if the backend is missing I used a Asynchronous approach, this is the create operation returns a Task ID, and it's up to the client to check if the task has been completed, normally it is desirable that the APP itself notifies the clients when their tasks have been completed, nevertheless due to the nature of the HTTP transport, this is not possible.

There are some tasks that require a synchronous response, these are: check the list of recipes and check the status of a given task. These calls have been implemented with JMS queues as well, but leveraged mule's request-reply router for synchronizing these calls, so if a redeployment of the backend happens in a reasonable time-frame, requests from the client will last a little longer but not fail.

Conclusions

The use of Mule ESB 3.5.0 allowed me to implement this use case in a matter of a couple of hours. In particular this version with the ability of sharing the JMS connector I can easily do Inter-APP communication (which is easier in Mule Enterprise by sharing a VM connector) and still have the ability to scale, add more servers, cluster etc.

Building Mule Applications with Gradle

Some time ago I discussed the advantages of building applications with gradle against maven or other build systems. I have been working since on a gradle plugin that enables Mule ESB users to build their apps with gradle as an alternative to maven.

While I continue to add features to this plugin, it is currently stable enough to build any community-based mule project and the plugin is open source based on the Apache 2 license. The url for the plugin and install instructions is the following:

https://github.com/mulesoft/mule-gradle-plugin

The project aims to cover all the features the current Mule plugin for maven does and some other use cases that might be difficult for the maven plugin to achieve. The plugin is oriented for both community and enterprise users of Mule ESB.

Using the Plugin With MuleStudio

In order to enable an existing MuleStudio project, you need to follow the following steps:

1. Install the gradle plugin into your plugin repositories. The README file suggest how to install it and use it in your local maven repo.
2. Create a MuleStudio project or choose the one you'd need to enable with gradle.
3. Add a build.gradle file in the root of your project with the following contents:

This base build script may be customized to include any dependencies your project may have but bare in mind that most of the libraries included in the Mule ESB runtime are already there in the 'provided' scope.

To upgrade your MuleStudio classpath automatically, you just need to run:

$ gradle studio

You need to run the previous command every time you update dependencies in your project. The mule version needs to match the version of the runtime you're using.

To build your app so it is deployable in a standalone studio, you just need to run:

$ gradle mulezip

This will place the deployable zip file in the build directory. The plugin is as well capable of running the app in an embedded mule instance or installing it into a Mule ESB instance installed on the same host.

Migration of a Maven-based project to Gradle

Recently I've started noticing that some of the big names in java open source have migrated their builds from Maven to Gradle.

To my own development needs, Maven was good enough, so for some time I remained sceptic with Gradle and carried on working with the former. Today I gave Gradle an opportunity and so far I find it has been amazing.

As with any new technology, to gain adoption you can't just simply be "better" because for most of the people, changing stuff require a lot of effort, so marginal improvements are simply not strong enough, you need to be radically better. I believe the creators of Gradle had this in mind when they created this tool.

The learning Curve

Any tool has it's own learning curve so, before I even tried to do something with it, I started reading the documentation. At the very beginning, it seemed to me that Gradle was just a pretty redo of Apache Ant but written in groovy and I started wondering why I was wasting my time on that. 

My advice for those who get the same initial feeling is: carry on reading, it's going to get good really soon.

After reading the quick-start chapter of the Gradle documentation I was able to see the power of this tool but still I had the taste that it was something equals to Ant but prettier and empowered by the fact that the builds are scripted in Groovy. The breakthrough came to me when I started reading about the so called "plugins", these plugins allow a full set of build conventions (like maven plugins) but are extremely simple to configure (unlike maven plugins).

After a little more than two hours of reading, I already had a couple ideas on how to use Gradle for building my apps, so I started migrating a simple tool I have for my day-to-day work. I'd need to say that the learning curve is very smooth.

The project I migrated

To start playing, I took one desktop-app I created to help my day-to-day work, this is just a simple Java based app, with Swing UI but it has an embedded instance of Mule ESB. This app was distributed as a zip file that anyone could execute with two clicks. The following is the pom.xml file I had to build and distribute the app.

I used an archetype that allowed me to easily get the executable file with its dependencies for running, also I had configured the project so it creates a zip file that I can distribute without much effort.

In order to start migrating the app, first I needed to create the Gradle build file, this is a groovy script called "build.gradle", this file is our new "pom.xml".

This is how the final file looks:

The first line of the script applies the java plugin to the project, this imports a predefined build configuration that applies to common java projects.

My project directory structure already had the "/src/main/java", "/src/main/resources", etc that typical maven projects have, I am happy with that structure and I simply didn't wish to change it, fortunately the creators of the java plugin liked these conventions as well and made them to be their standard.

The second line of the script applies the "application" plugin, this is a fantastic plugin that helps you distribute standalone java apps.

Next, I define the naming properties of my project, this is the same concept of the coordinates we need to define for each maven project, the main difference is that these are optional. The name or the artifact will be taken from the name of the folder where the project is contained.

Then, I defined as extended properties the versions of each of my dependencies, my first attempt was to not use the 'ext.' object but Gradle printed a warning when building saying that this is deprecated, It was not obvious to me how to get this done properly, so I took a sneak peak on the spring framework build script to get some inspiration.

After that, I've defined the libraries my project depends on. Gradle provides a more concise syntax for defining this as well but I sticked with the little verbosity that I'm used to because of my maven background.

I configured the main class as a project property as well, this is just required for the application plugin to know what is the entry point of the app.

Finally, I've configured my repos. Since Gradle can use maven repos (but it is not maven) I need to specify that I want the maven central repo to be included, as well as my local repo (.m2/repository), and a couple of other repos I normally configure.

That is all for the migration of my build, let's now take a look at how to use it!

Building and Distributing

To build the project I can simply run:

$ gradle build

The 'build' word is just one of the tasks the java plugin for Gradle has. We have many options i.e. clean, test, etc, just like we would expect from Maven out of the box. For a list of tasks we can run, please take a look at:

Gradle Java Plugin

Also I can use tasks from the application plugin to run my app:

$ gradle run

And this runs the app from the defined main class. This application plugin has also some very interesting options for me to distribute this app, for example I could create a distribution zip file, tar file, and even os-specific startup scripts, for more information on what you can do with this plugin, just take a look at:

Gradle Application Plugin

Conclusions

This configuration is just scratching the surface of what you can do with this very powerful build system and also since this is just my first ride, there should be things that I could have done better, for example maybe there is a way to define the default set of repositories for each build.

My overall feeling with this build system is that the configuration is more concise, and making complex configurations that would normally take creating Mojos and a lot of reading, can be simply done with a small amount of groovy.

Even though I din't get the exact same result in my distribution, I like what I had out of the box. My previous distribution was based as well on accepting what I got from an archetype, so in this case, I'd need to say that what comes out of the box is good enough.

What I didn't like is that maven does not regard itself as a mere build tool but it is aimed to standardize projects, I've got the feeling that this idea has been lost on Gradle.

Gradle has surprised me with its simplicity and yet it looks like you can tackle simple and complex things in the same way, this is, of course, thanks to the pre-built plugins, that are, I believe, what makes the difference for the adoption of this tool.

How To Unit-Test an Annotation Processor

The Issue

I have been developing lately an annotation processor and found that unit-testing it is not as trivial as it might sound.

Some would say "You just need to mock all of the java.lang.model.* packages and that's it!" Unfortunately mocking these elements is not as easy as it might appear and I don't really find a reason why to do this other than speeding up the test suite, this alternative has to be taken real seriously and prioritize the test time / development effort.

The solution

The path I went through is invoking the Java compiler within my unit tests. Doing this is kind of straightforward but some considerations must be taken.

• In order for the tests to be repeatable, please make sure your working directory is invariant, maven does a good job at this.
• It's easier if your annotation processor can be auto-discovered. This is place the appropriate "javax.annotation.processing.Processor" file on META-INF/services/
• Make sure you delete the junk files after you have tested. Since we're going to compile classes some .class files may appear, it is not mandatory to delete them but they would get mixed with the source code which is not nice at all.

Now that we've taken care of the errands, I present you a generic annotation processing test case which you can use for your unit tests.

Please note that this is not the only (or strictly the best) way of doing this but is a way and I like it so, here is the code:

First, we have a simple interface which we will implement to build our test cases, implementations return the list of files to compile and then might verify if the compilation has failed of there were some compilation messages.

Next, we have a really simple sample implementation of a smoke test which verifies that no "Critical Warnings" or "Compiler Errors" have been issued from a valid configuration.

Finally we have the test class. For this I make use of the parameterized test runner that JUnit 4 provides and the parameters are exactly the different implementations of our simple interface.

Please note that in order to be completely generic, this unit test must obtain its parameters from configuration nevertheless I made it like this for the sake of simplicity and just keeping note of the room for improvement.

After this it is just boilerplate code, get the compiler, configure the file manager, compiling the relevant files, invoking the test and finally cleaning up after ourselves.

Please feel free to modify or upgrade this test case and I hope you find it as useful as it is to me. 

jDTO Binder gains experimental support for compile-time config validation

One of the most frustrating things that emerge when working with a framework is misconfiguration, specially if we find out of this misconfiguration on runtime or worse, after moving the application into production.

Normally when frameworks suffer this kind of weakness, developers might decide to create IDE-specific plugins or extra tooling to prevent unwanted and unnoticed runtime misconfiguration. The object to object mapping framework I've developed (jDTO Binder) is a great tool but sadly suffers from this issue so, as part of release 1.5, I am currently developing a set of tools that aim to ease the development. The first of this tool is a compile-time annotation processor that can validate the configuration for a given DTO.

Annotation Config Validation

The great thing of compile-time annotation processors is that are able to integrate with any IDE and they're able to have the build failing if a misconfiguration is detected.

Here is how it looks jDTO Binder compile-time verifier out of the box in Netbeans IDE 7.2:

Currently this is experimental and I'm working on the idea, the specific validations that will be performed and the way this validations should manifest will emerge on the development of this idea and for the time being very experimental and limited support is implemented but hopefully it will plot on improved productivity.

In the current state the compile time verifier functionality checks and prints compile-time warnings when the source property of a given DTO field is mistyped and also encourages the user to place annotations on getters instead of on fields following a performance suggestion showed on the official documentation.

Netbeans-Like Distribution with Maven

When it comes to creating standalone Java applications, out of the box maven projects are not easy to distribute, they don't have configured some default main class (or some convention), dependencies are not specified on the manifest and also dependency jars are not packaged even near the final jar file.

On the other hand, if you have used Netbeans IDE to create standalone Java apps, you can notice that upon any build, the default Netbeans ant build script places a "dist" folder inside your project with the following structure:

This folder is the distribution of your app and you may simply run it with the java -jar command line.

Even though Maven projects have a lot of advantages, I personally miss how handy it is this structure for distribution; so I managed to make this happen with maven and it's assembly plugin.

Building the Distribution Structure

In order to create the desired distribution, I started with maven's jar-with-dependencies predefined assembly and customized it to look like the following:

What this does is the following:

• It compresses the whole distribution into a nice zip file inside the target directory.
• The first dependency set is there to copy the application jar on the root of the distribution, I added the none scope to make sure that no other dependencies are copied there.
• The second dependency set is to copy all dependencies and transitive dependencies into the lib folder.

Now this takes care of the packaging but there is still some work to do, so we need to edit our project's pom.xml file and add the following configurations:

First of all, I've changed the final name of the archive to be short and sweet. Next, I've configured the assembly plugin to point to the right configuration file. Finally I've configured the jar plugin to set in the MANIFEST.MF file the main class and the jars that should be included on the classpath. Please note that you should alter the mainClass configuration to match you main class.

After this, we can create our distribution by simply running:

$ mvn clean package assembly:single

You should find a zip file called like your artifact's ID-dist.zip, which you can nicely share and it's easily run.

Getting Started With Apache Maven

A couple of days ago, I spoke on a tech talk of how to get started with apache maven. Some of you may know and use maven and some of you don't.

I've created a presentation explaining some things about maven, specially why to use it and what are the problems it solves, which you can of course download.

Also I've recorded a video on how to get started by creating a project, running it, and exploring how eclipse and netbeans can work with maven projects.

Hope you find it useful.

Setting up maven to use local JAR Libraries

Since the very first time I've started using maven (and as a previous ANT user) I wondered how I could achieve the only thing that ANT does great: Add manually my own libraries.

The answer is easy: install them into the local repository. But that's not so easy! If someone new joined the project, he would have to install the artifact also in his local repository and this is an extra step anyone could easily forget!

Another thing you could do is install nexus in some tomcat, and after a little bit of configuration install the jar and add the server url as a repository on my project, but... Isn't that a little bit of overkill for one or two JARS?

All the previous analysis already contains the solution: we need to do both: Add a repository, but the repository should be local... to the project and manually install the files there (and probably check them in into a repository).

And here is how you do It: First of all you need to know that you can add a file:// url as the url of the repository, so we can add something like this:

   <repositories> 
    <repository> 
      <id>project</id> 
      <name>Project Maven Repository</name> 
      <layout>default</layout> 
      <url>file://${project.basedir}/lib/</url> 
    </repository> 
  </repositories> 

And we take advantage of the variable referencing the project's directory: project.basedir.

So now we only need to copy artifacts installed manually from our local repository to our project repository (or better install them directly into the project repository). And that's it your project build will still be flawless and you won't have to worry about putting nexus to work or remind your new co-workers to install the artifact.