Sunday, September 28, 2014

Embedded Jetty and Apache CXF: secure REST services with Spring Security

Recently I run into very interesting problem which I thought would take me just a couple of minutes to solve: protecting Apache CXF (current release 3.0.1)/ JAX-RS REST services with Spring Security (current stable version 3.2.5) in the application running inside embedded Jetty container (current release 9.2). At the end, it turns out to be very easy, once you understand how things work together and known subtle intrinsic details. This blog post will try to reveal that.

Our example application is going to expose a simple JAX-RS / REST service to manage people. However, we do not want everyone to be allowed to do that so the HTTP basic authentication will be required in order to access our endpoint, deployed at http://localhost:8080/api/rest/people. Let us take a look on the PeopleRestService class:

package com.example.rs;

import javax.json.Json;
import javax.json.JsonArray;
import javax.ws.rs.GET;
import javax.ws.rs.Path;
import javax.ws.rs.Produces;

@Path( "/people" ) 
public class PeopleRestService {
    @Produces( { "application/json" } )
    @GET
    public JsonArray getPeople() {
        return Json.createArrayBuilder()
            .add( Json.createObjectBuilder()
                .add( "firstName", "Tom" )
                .add( "lastName", "Tommyknocker" )
                .add( "email", "a@b.com" ) )
            .build();
    }
}

As you can see in the snippet above, nothing is pointing out to the fact that this REST service is secured, just couple of familiar JAX-RS annotations.

Now, let us declare the desired security configuration following excellent Spring Security documentation. There are many ways to configure Spring Security but we are going to show off two of them: using in-memory authentication and using user details service, both built on top of WebSecurityConfigurerAdapter. Let us start with in-memory authentication as it is the simplest one:

package com.example.config;

import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.context.annotation.Configuration;
import org.springframework.security.config.annotation.authentication.builders.AuthenticationManagerBuilder;
import org.springframework.security.config.annotation.method.configuration.EnableGlobalMethodSecurity;
import org.springframework.security.config.annotation.web.builders.HttpSecurity;
import org.springframework.security.config.annotation.web.configuration.EnableWebSecurity;
import org.springframework.security.config.annotation.web.configuration.WebSecurityConfigurerAdapter;
import org.springframework.security.config.http.SessionCreationPolicy;

@Configuration
@EnableWebSecurity
@EnableGlobalMethodSecurity( securedEnabled = true )
public class InMemorySecurityConfig extends WebSecurityConfigurerAdapter {
    @Autowired
    public void configureGlobal(AuthenticationManagerBuilder auth) throws Exception {
        auth.inMemoryAuthentication()
            .withUser( "user" ).password( "password" ).roles( "USER" ).and()
            .withUser( "admin" ).password( "password" ).roles( "USER", "ADMIN" );
    }

    @Override
    protected void configure( HttpSecurity http ) throws Exception {
        http.httpBasic().and()
            .sessionManagement().sessionCreationPolicy( SessionCreationPolicy.STATELESS ).and()
            .authorizeRequests().antMatchers("/**").hasRole( "USER" );
    }
}

In the snippet above there two users defined: user with the role USER and admin with the roles USER, ADMIN. We also protecting all URLs (/**) by setting authorization policy to allow access only users with role USER. Being just a part of the application configuration, let us plug it into the AppConfig class using @Import annotation.

package com.example.config;

import java.util.Arrays;

import javax.ws.rs.ext.RuntimeDelegate;

import org.apache.cxf.bus.spring.SpringBus;
import org.apache.cxf.endpoint.Server;
import org.apache.cxf.jaxrs.JAXRSServerFactoryBean;
import org.apache.cxf.jaxrs.provider.jsrjsonp.JsrJsonpProvider;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.context.annotation.DependsOn;
import org.springframework.context.annotation.Import;

import com.example.rs.JaxRsApiApplication;
import com.example.rs.PeopleRestService;

@Configuration
@Import( InMemorySecurityConfig.class )
public class AppConfig { 
    @Bean( destroyMethod = "shutdown" )
    public SpringBus cxf() {
        return new SpringBus();
    }
 
    @Bean @DependsOn ( "cxf" )
    public Server jaxRsServer() {
        JAXRSServerFactoryBean factory = RuntimeDelegate.getInstance().createEndpoint( jaxRsApiApplication(), JAXRSServerFactoryBean.class );
        factory.setServiceBeans( Arrays.< Object >asList( peopleRestService() ) );
        factory.setAddress( factory.getAddress() );
        factory.setProviders( Arrays.< Object >asList( new JsrJsonpProvider() ) );
        return factory.create();
    }
 
    @Bean 
    public JaxRsApiApplication jaxRsApiApplication() {
        return new JaxRsApiApplication();
    }
 
    @Bean 
    public PeopleRestService peopleRestService() {
        return new PeopleRestService();
    }  
}

At this point we have all the pieces except the most interesting one: the code which runs embedded Jetty instance and creates proper servlet mappings, listeners, passing down the configuration we have created.

package com.example;

import java.util.EnumSet;

import javax.servlet.DispatcherType;

import org.apache.cxf.transport.servlet.CXFServlet;
import org.eclipse.jetty.server.Server;
import org.eclipse.jetty.servlet.FilterHolder;
import org.eclipse.jetty.servlet.ServletContextHandler;
import org.eclipse.jetty.servlet.ServletHolder;
import org.springframework.web.context.ContextLoaderListener;
import org.springframework.web.context.support.AnnotationConfigWebApplicationContext;
import org.springframework.web.filter.DelegatingFilterProxy;

import com.example.config.AppConfig;

public class Starter {
    public static void main( final String[] args ) throws Exception {
        Server server = new Server( 8080 );
          
        // Register and map the dispatcher servlet
        final ServletHolder servletHolder = new ServletHolder( new CXFServlet() );
        final ServletContextHandler context = new ServletContextHandler();   
        context.setContextPath( "/" );
        context.addServlet( servletHolder, "/rest/*" );  
        context.addEventListener( new ContextLoaderListener() );
   
        context.setInitParameter( "contextClass", AnnotationConfigWebApplicationContext.class.getName() );
        context.setInitParameter( "contextConfigLocation", AppConfig.class.getName() );
   
        // Add Spring Security Filter by the name
        context.addFilter(
            new FilterHolder( new DelegatingFilterProxy( "springSecurityFilterChain" ) ), 
                "/*", EnumSet.allOf( DispatcherType.class )
        );
         
        server.setHandler( context );
        server.start();
        server.join(); 
    }
}

Most of the code does not require any explanation except the the filter part. This is what I meant by subtle intrinsic detail: the DelegatingFilterProxy should be configured with the filter name which must be exactly springSecurityFilterChain, as Spring Security names it. With that, the security rules we have configured are going to apply to any JAX-RS service call (the security filter is executed before the Apache CXF servlet), requiring the full authentication. Let us quickly check that by building and running the project:

mvn clean package   
java -jar target/jax-rs-2.0-spring-security-0.0.1-SNAPSHOT.jar

Issuing the HTTP GET call without providing username and password does not succeed and returns HTTP status code 401.

> curl -i http://localhost:8080/rest/api/people

HTTP/1.1 401 Full authentication is required to access this resource
WWW-Authenticate: Basic realm="Realm"
Cache-Control: must-revalidate,no-cache,no-store
Content-Type: text/html; charset=ISO-8859-1
Content-Length: 339
Server: Jetty(9.2.2.v20140723)

The same HTTP GET call with username and password provided returns successful response (with some JSON generated by the server).

> curl -i -u user:password http://localhost:8080/rest/api/people

HTTP/1.1 200 OK
Date: Sun, 28 Sep 2014 20:07:35 GMT
Content-Type: application/json
Content-Length: 65
Server: Jetty(9.2.2.v20140723)

[{"firstName":"Tom","lastName":"Tommyknocker","email":"a@b.com"}]

Excellent, it works like a charm! Turns out, it is really very easy. Also, as it was mentioned before, the in-memory authentication could be replaced with user details service, here is an example how it could be done:

package com.example.config;

import java.util.Arrays;

import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.security.config.annotation.authentication.builders.AuthenticationManagerBuilder;
import org.springframework.security.config.annotation.method.configuration.EnableGlobalMethodSecurity;
import org.springframework.security.config.annotation.web.builders.HttpSecurity;
import org.springframework.security.config.annotation.web.configuration.EnableWebSecurity;
import org.springframework.security.config.annotation.web.configuration.WebSecurityConfigurerAdapter;
import org.springframework.security.config.http.SessionCreationPolicy;
import org.springframework.security.core.authority.SimpleGrantedAuthority;
import org.springframework.security.core.userdetails.User;
import org.springframework.security.core.userdetails.UserDetails;
import org.springframework.security.core.userdetails.UserDetailsService;
import org.springframework.security.core.userdetails.UsernameNotFoundException;

@Configuration
@EnableWebSecurity
@EnableGlobalMethodSecurity(securedEnabled = true)
public class UserDetailsSecurityConfig extends WebSecurityConfigurerAdapter {
    @Autowired
    public void configureGlobal(AuthenticationManagerBuilder auth) throws Exception {
        auth.userDetailsService( userDetailsService() );
    }
    
    @Bean
    public UserDetailsService userDetailsService() {
        return new UserDetailsService() {
            @Override
            public UserDetails loadUserByUsername( final String username ) 
                    throws UsernameNotFoundException {
                if( username.equals( "admin" ) ) {
                    return new User( username, "password", true, true, true, true,
                        Arrays.asList(
                            new SimpleGrantedAuthority( "ROLE_USER" ),
                            new SimpleGrantedAuthority( "ROLE_ADMIN" )
                        )
                    );
                } else if ( username.equals( "user" ) ) {
                    return new User( username, "password", true, true, true, true,
                        Arrays.asList(
                            new SimpleGrantedAuthority( "ROLE_USER" )
                        )
                    );
                } 
                    
                return null;
            }
        };
    }

    @Override
    protected void configure( HttpSecurity http ) throws Exception {
        http
           .httpBasic().and()
           .sessionManagement().sessionCreationPolicy( SessionCreationPolicy.STATELESS ).and()
           .authorizeRequests().antMatchers("/**").hasRole( "USER" );
    }
}

Replacing the @Import( InMemorySecurityConfig.class ) with @Import( UserDetailsSecurityConfig.class ) in the AppConfig class leads to the same results, as both security configurations define the identical sets of users and their roles.

I hope, this blog post will save you some time and gives a good starting point, as Apache CXF and Spring Security are getting along very well under Jetty umbrella!

The complete source code is available on GitHub.

Saturday, August 9, 2014

OSGi: the gateway into micro-services architecture

The terms "modularity" and "microservices architecture" pop up quite often these days in context of building scalable, reliable distributed systems. Java platform itself is known to be weak with regards to modularity (Java 9 is going to address that by delivering project Jigsaw), giving a chance to frameworks like OSGi and JBoss Modules to emerge.

When I first heard about OSGi back in 2007, I was truly excited about all those advantages Java applications might benefit of by being built on top of it. But very quickly the frustration took place instead of excitement: no tooling support, very limited set of compatible libraries and frameworks, quite unstable and hard to troubleshoot runtime. Clearly, it was not ready to be used by average Java developer and as such, I had to put it on the shelf. With years, OSGi has matured a lot and gained a widespread community support.

The curious reader may ask: what are the benefits of using modules and OSGi in particular? To name just a few problems it helps to solve:

  • explicit (and versioned) dependency management: modules declare what they need (and optionally the version ranges)
  • small footprint: modules are not packaged with all their dependencies
  • easy release: modules can be developed and released independently
  • hot redeploy: individual modules may be redeployed without affecting others

In today's post we are going to take a 10000 feet view on a state of the art in building modular Java applications using OSGi. Leaving aside discussions how good or bad OSGi is, we are going to build an example application consisting of following modules:

  • data access module
  • business services module
  • REST services module
Apache OpenJPA 2.3.0 / JPA 2.0 for data access (unfortunately, JPA 2.1 is not yet supported by OSGi implementation of our choice), Apache CXF 3.0.1 / JAX-RS 2.0 for REST layer are two main building blocks of the application. I found Christian Schneider's blog, Liquid Reality, to be invaluable source of information about OSGi (as well as many other topics).

In OSGi world, the modules are called bundles. Bundles manifest their dependencies (import packages) and the packages they expose (export packages) so other bundles are able to use them. Apache Maven supports this packaging model as well. The bundles are managed by OSGi runtime, or container, which in our case is going to be Apache Karaf 3.0.1 (actually, it is the single thing we need to download and unpack).

Let me stop talking and better show some code. We are going to start from the top (REST) and go all the way to the bottom (data access) as it would be easier to follow. Our PeopleRestService is a typical example of JAX-RS 2.0 service implementation:

package com.example.jaxrs;

import java.util.Collection;

import javax.ws.rs.DELETE;
import javax.ws.rs.DefaultValue;
import javax.ws.rs.FormParam;
import javax.ws.rs.GET;
import javax.ws.rs.POST;
import javax.ws.rs.PUT;
import javax.ws.rs.Path;
import javax.ws.rs.PathParam;
import javax.ws.rs.Produces;
import javax.ws.rs.QueryParam;
import javax.ws.rs.core.Context;
import javax.ws.rs.core.MediaType;
import javax.ws.rs.core.Response;
import javax.ws.rs.core.UriInfo;

import com.example.data.model.Person;
import com.example.services.PeopleService;

@Path( "/people" )
public class PeopleRestService {
    private PeopleService peopleService;
        
    @Produces( { MediaType.APPLICATION_JSON } )
    @GET
    public Collection< Person > getPeople( 
            @QueryParam( "page") @DefaultValue( "1" ) final int page ) {
        return peopleService.getPeople( page, 5 );
    }

    @Produces( { MediaType.APPLICATION_JSON } )
    @Path( "/{email}" )
    @GET
    public Person getPerson( @PathParam( "email" ) final String email ) {
        return peopleService.getByEmail( email );
    }

    @Produces( { MediaType.APPLICATION_JSON  } )
    @POST
    public Response addPerson( @Context final UriInfo uriInfo,
            @FormParam( "email" ) final String email, 
            @FormParam( "firstName" ) final String firstName, 
            @FormParam( "lastName" ) final String lastName ) {
        
        peopleService.addPerson( email, firstName, lastName );
        return Response.created( uriInfo
            .getRequestUriBuilder()
            .path( email )
            .build() ).build();
    }
    
    @Produces( { MediaType.APPLICATION_JSON  } )
    @Path( "/{email}" )
    @PUT
    public Person updatePerson( @PathParam( "email" ) final String email, 
            @FormParam( "firstName" ) final String firstName, 
            @FormParam( "lastName" )  final String lastName ) {
        
        final Person person = peopleService.getByEmail( email );
        
        if( firstName != null ) {
            person.setFirstName( firstName );
        }
        
        if( lastName != null ) {
            person.setLastName( lastName );
        }

        return person;              
    }
    
    @Path( "/{email}" )
    @DELETE
    public Response deletePerson( @PathParam( "email" ) final String email ) {
        peopleService.removePerson( email );
        return Response.ok().build();
    }
    
    public void setPeopleService( final PeopleService peopleService ) {
        this.peopleService = peopleService;
    }
}

As we can see, there is nothing here telling us about OSGi. The only dependency is the PeopleService which somehow should be injected into the PeopleRestService. How? Typically, OSGi applications use blueprint as the dependency injection framework, very similar to old buddy, XML based Spring configuration. It should be packaged along with application inside OSGI-INF/blueprint folder. Here is a blueprint example for our REST module, built on top of Apache CXF 3.0.1:

<blueprint xmlns="http://www.osgi.org/xmlns/blueprint/v1.0.0"
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xmlns:jaxrs="http://cxf.apache.org/blueprint/jaxrs"
    xmlns:cxf="http://cxf.apache.org/blueprint/core"
    xsi:schemaLocation="
        http://www.osgi.org/xmlns/blueprint/v1.0.0 
        http://www.osgi.org/xmlns/blueprint/v1.0.0/blueprint.xsd
        http://cxf.apache.org/blueprint/jaxws 
        http://cxf.apache.org/schemas/blueprint/jaxws.xsd
        http://cxf.apache.org/blueprint/jaxrs 
        http://cxf.apache.org/schemas/blueprint/jaxrs.xsd
        http://cxf.apache.org/blueprint/core 
        http://cxf.apache.org/schemas/blueprint/core.xsd">

    <cxf:bus id="bus">
        <cxf:features>
            <cxf:logging/>
        </cxf:features>       
    </cxf:bus>

    <jaxrs:server address="/api" id="api">
        <jaxrs:serviceBeans>
            <ref component-id="peopleRestService"/>
        </jaxrs:serviceBeans>
        <jaxrs:providers>
            <bean class="com.fasterxml.jackson.jaxrs.json.JacksonJsonProvider" />
        </jaxrs:providers>
    </jaxrs:server>
    
    <!-- Implementation of the rest service -->
    <bean id="peopleRestService" class="com.example.jaxrs.PeopleRestService">
        <property name="peopleService" ref="peopleService"/>
    </bean>         
    
    <reference id="peopleService" interface="com.example.services.PeopleService" />
</blueprint>

Very small and simple: basically the configuration just states that in order for the module to work, the reference to the com.example.services.PeopleService should be provided (effectively, by OSGi container). To see how it is going to happen, let us take a look on another module which exposes services. It contains only one interface PeopleService:

package com.example.services;

import java.util.Collection;

import com.example.data.model.Person;

public interface PeopleService {
    Collection< Person > getPeople( int page, int pageSize );
    Person getByEmail( final String email );
    Person addPerson( final String email, final String firstName, final String lastName );
    void removePerson( final String email );
}

And also provides its implementation as PeopleServiceImpl class:

package com.example.services.impl;

import java.util.Collection;

import org.osgi.service.log.LogService;

import com.example.data.PeopleDao;
import com.example.data.model.Person;
import com.example.services.PeopleService;

public class PeopleServiceImpl implements PeopleService {
    private PeopleDao peopleDao;
    private LogService logService;
 
    @Override
    public Collection< Person > getPeople( final int page, final int pageSize ) {        
        logService.log( LogService.LOG_INFO, "Getting all people" );
        return peopleDao.findAll( page, pageSize );
    }

    @Override
    public Person getByEmail( final String email ) {
        logService.log( LogService.LOG_INFO, 
            "Looking for a person with e-mail: " + email );
        return peopleDao.find( email );        
    }

    @Override
    public Person addPerson( final String email, final String firstName, 
            final String lastName ) {
        logService.log( LogService.LOG_INFO, 
            "Adding new person with e-mail: " + email );
        return peopleDao.save( new Person( email, firstName, lastName ) );
    }

    @Override
    public void removePerson( final String email ) {
        logService.log( LogService.LOG_INFO, 
            "Removing a person with e-mail: " + email );
        peopleDao.delete( email );
    }
    
    public void setPeopleDao( final PeopleDao peopleDao ) {
        this.peopleDao = peopleDao;
    }
    
    public void setLogService( final LogService logService ) {
        this.logService = logService;
    }
}

And this time again, very small and clean implementation with two injectable dependencies, org.osgi.service.log.LogService and com.example.data.PeopleDao. Its blueprint configuration, located inside OSGI-INF/blueprint folder, looks quite compact as well:

<blueprint xmlns="http://www.osgi.org/xmlns/blueprint/v1.0.0"  
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"         
    xsi:schemaLocation="
        http://www.osgi.org/xmlns/blueprint/v1.0.0 
        http://www.osgi.org/xmlns/blueprint/v1.0.0/blueprint.xsd">
        
    <service ref="peopleService" interface="com.example.services.PeopleService" />        
    <bean id="peopleService" class="com.example.services.impl.PeopleServiceImpl">
        <property name="peopleDao" ref="peopleDao" />    
        <property name="logService" ref="logService" />
    </bean>
    
    <reference id="peopleDao" interface="com.example.data.PeopleDao" />
    <reference id="logService" interface="org.osgi.service.log.LogService" />
</blueprint>

The references to PeopleDao and LogService are expected to be provided by OSGi container at runtime. Hovewer, PeopleService implementation is exposed as service and OSGi container will be able to inject it into PeopleRestService once its bundle is being activated.

The last piece of the puzzle, data access module, is a bit more complicated: it contains persistence configuration (META-INF/persistence.xml) and basically depends on JPA 2.0 capabilities of the OSGi container. The persistence.xml is quite basic:

<persistence xmlns="http://java.sun.com/xml/ns/persistence"
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    version="2.0">
 
    <persistence-unit name="peopleDb" transaction-type="JTA">
        <jta-data-source>
        osgi:service/javax.sql.DataSource/(osgi.jndi.service.name=peopleDb)
        </jta-data-source>       
        <class>com.example.data.model.Person</class>
        
        <properties>
            <property name="openjpa.jdbc.SynchronizeMappings" 
                value="buildSchema"/>         
        </properties>        
    </persistence-unit>
</persistence>

Similarly to the service module, there is an interface PeopleDao exposed:

package com.example.data;

import java.util.Collection;

import com.example.data.model.Person;

public interface PeopleDao {
    Person save( final Person person );
    Person find( final String email );
    Collection< Person > findAll( final int page, final int pageSize );
    void delete( final String email ); 
}

With its implementation PeopleDaoImpl:

package com.example.data.impl;

import java.util.Collection;

import javax.persistence.EntityManager;
import javax.persistence.criteria.CriteriaBuilder;
import javax.persistence.criteria.CriteriaQuery;

import com.example.data.PeopleDao;
import com.example.data.model.Person;

public class PeopleDaoImpl implements PeopleDao {
    private EntityManager entityManager;
 
    @Override
    public Person save( final Person person ) {
        entityManager.persist( person );
        return person;
    }
 
    @Override
    public Person find( final String email ) {
        return entityManager.find( Person.class, email );
    }
 
    public void setEntityManager( final EntityManager entityManager ) {
        this.entityManager = entityManager;
    }
 
    @Override
    public Collection< Person > findAll( final int page, final int pageSize ) {
        final CriteriaBuilder cb = entityManager.getCriteriaBuilder();
     
        final CriteriaQuery< Person > query = cb.createQuery( Person.class );
        query.from( Person.class );
     
        return entityManager
            .createQuery( query )
            .setFirstResult(( page - 1 ) * pageSize )
            .setMaxResults( pageSize ) 
            .getResultList();
    }
 
    @Override
    public void delete( final String email ) {
        entityManager.remove( find( email ) );
    }
}

Please notice, although we are performing data manipulations, there is no mention of transactions as well as there are no explicit calls to entity manager's transactions API. We are going to use the declarative approach to transactions as blueprint configuration supports that (the location is unchanged, OSGI-INF/blueprint folder):

<blueprint xmlns="http://www.osgi.org/xmlns/blueprint/v1.0.0"  
    xmlns:jpa="http://aries.apache.org/xmlns/jpa/v1.1.0"
    xmlns:tx="http://aries.apache.org/xmlns/transactions/v1.0.0" 
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"         
    xsi:schemaLocation="
        http://www.osgi.org/xmlns/blueprint/v1.0.0 
        http://www.osgi.org/xmlns/blueprint/v1.0.0/blueprint.xsd">
    
    <service ref="peopleDao" interface="com.example.data.PeopleDao" />
    <bean id="peopleDao" class="com.example.data.impl.PeopleDaoImpl">
     <jpa:context unitname="peopleDb" property="entityManager" />
     <tx:transaction method="*" value="Required"/>
    </bean>
    
    <bean id="dataSource" class="org.hsqldb.jdbc.JDBCDataSource">
       <property name="url" value="jdbc:hsqldb:mem:peopleDb"/>
    </bean>
    
    <service ref="dataSource" interface="javax.sql.DataSource"> 
        <service-properties> 
            <entry key="osgi.jndi.service.name" value="peopleDb" /> 
        </service-properties> 
    </service>     
</blueprint>

One thing to keep in mind: the application doesn't need to create JPA 2.1's entity manager: the OSGi runtime is able do that and inject it everywhere it is required, driven by jpa:context declarations. Consequently, tx:transaction instructs the runtime to wrap the selected service methods inside transaction.

Now, when the last service PeopleDao is exposed, we are ready to deploy our modules with Apache Karaf 3.0.1. It is quite easy to do in three steps:

  • run the Apache Karaf 3.0.1 container
    bin/karaf (or bin\karaf.bat on Windows)
    
  • execute following commands from the Apache Karaf 3.0.1 shell:
    feature:repo-add cxf 3.0.1
    feature:install http cxf jpa openjpa transaction jndi jdbc
    install -s mvn:org.hsqldb/hsqldb/2.3.2
    install -s mvn:com.fasterxml.jackson.core/jackson-core/2.4.0
    install -s mvn:com.fasterxml.jackson.core/jackson-annotations/2.4.0
    install -s mvn:com.fasterxml.jackson.core/jackson-databind/2.4.0
    install -s mvn:com.fasterxml.jackson.jaxrs/jackson-jaxrs-base/2.4.0
    install -s mvn:com.fasterxml.jackson.jaxrs/jackson-jaxrs-json-provider/2.4.0
    
  • build our modules and copy them into Apache Karaf 3.0.1's deploy folder (while container is still running):
    mvn clean package
    cp module*/target/*jar apache-karaf-3.0.1/deploy/
    
When you run the list command in the Apache Karaf 3.0.1 shell, you should see the list of all activated bundles (modules), similar to this one:

Where module-service, module-jax-rs and module-data correspond to the ones we are being developed. By default, all our Apache CXF 3.0.1 services will be available at base URL http://:8181/cxf/api/. It is easy to check by executing cxf:list-endpoints -f command in the Apache Karaf 3.0.1 shell.

Let us make sure our REST layer works as expected by sending couple of HTTP requests. Let us create new person:

curl http://localhost:8181/cxf/api/people -iX POST -d "firstName=Tom&lastName=Knocker&email=a@b.com"

HTTP/1.1 201 Created
Content-Length: 0
Date: Sat, 09 Aug 2014 15:26:17 GMT
Location: http://localhost:8181/cxf/api/people/a@b.com
Server: Jetty(8.1.14.v20131031)

And verify that person has been created successfully:

curl -i http://localhost:8181/cxf/api/people

HTTP/1.1 200 OK
Content-Type: application/json
Date: Sat, 09 Aug 2014 15:28:20 GMT
Transfer-Encoding: chunked
Server: Jetty(8.1.14.v20131031)

[{"email":"a@b.com","firstName":"Tom","lastName":"Knocker"}]

Would be nice to check if database has the person populated as well. With Apache Karaf 3.0.1 shell it is very simple to do by executing just two commands: jdbc:datasources and jdbc:query peopleDb "select * from people".

Awesome! I hope this quite introductory blog post opens yet another piece of interesting technology you may use for developing robust, scalable, modular and manageable software. We have not touched many, many things but these are here for you to discover. The complete source code is available on GitHub.

Note to Hibernate 4.2.x / 4.3.x users: unfortunately, in the current release of Apache Karaf 3.0.1 the Hibernate 4.3.x does work properly at all (as JPA 2.1 is not yet supported) and, however I have managed to run with Hibernate 4.2.x, the container often refused to resolve the JPA-related dependencies.

Thursday, May 29, 2014

Apache CXF 3.0: CDI 1.1 support as alternative to Spring

With Apache CXF 3.0 just being released a couple of weeks ago, the project makes yet another important step to fulfill the JAX-RS 2.0 specification requirements: integration with CDI 1.1. In this blog post we are going to look on a couple of examples of how Apache CXF 3.0 and Apache CXF 3.0 work together.

Starting from version 3.0, Apache CXF includes a new module, named cxf-integration-cdi which could be added easily to your Apache Maven POM file:

<dependency>
    <groupId>org.apache.cxf</groupId>
    <artifactId>cxf-integration-cdi</artifactId>
    <version>3.0.0</version>
</dependency>

This new module brings just two components (in fact, a bit more but those are the key ones):

  • CXFCdiServlet: the servlet to bootstrap Apache CXF application, serving the same purpose as CXFServlet and CXFNonSpringJaxrsServlet, ...
  • JAXRSCdiResourceExtension: portable CDI 1.1 extension where all the magic happens
When run in CDI 1.1-enabled environment, the portable extensions are discovered by CDI 1.1 container and initialized using life-cycle events. And that is literally all what you need! Let us see the real application in action.

We are going to build a very simple JAX-RS 2.0 application to manage people using Apache CXF 3.0 and JBoss Weld 2.1, the CDI 1.1 reference implementation. The Person class we are going to use for a person representation is just a simple Java bean:

package com.example.model;

public class Person {
    private String email;
    private String firstName;
    private String lastName;
  
    public Person() {
    }
 
    public Person( final String email, final String firstName, final String lastName ) {
        this.email = email;
        this.firstName = firstName;
        this.lastName = lastName;
    }

    // Getters and setters are ommited
    // ... 
}

As it is quite common now, we are going to run our application inside embedded Jetty 9.1 container and our Starter class does exactly that:

package com.example;

import org.apache.cxf.cdi.CXFCdiServlet;
import org.eclipse.jetty.server.Server;
import org.eclipse.jetty.servlet.ServletContextHandler;
import org.eclipse.jetty.servlet.ServletHolder;
import org.jboss.weld.environment.servlet.BeanManagerResourceBindingListener;
import org.jboss.weld.environment.servlet.Listener;

public class Starter { 
    public static void main( final String[] args ) throws Exception {
        final Server server = new Server( 8080 );
          
        // Register and map the dispatcher servlet
        final ServletHolder servletHolder = new ServletHolder( new CXFCdiServlet() );
        final ServletContextHandler context = new ServletContextHandler();   
        context.setContextPath( "/" );    
        context.addEventListener( new Listener() );   
        context.addEventListener( new BeanManagerResourceBindingListener() );
        context.addServlet( servletHolder, "/rest/*" );
   
        server.setHandler( context );
        server.start();        
        server.join(); 
    }
}

Please notice the presence of CXFCdiServlet and two mandatory listeners which were added to the context:

  • org.jboss.weld.environment.servlet.Listener is responsible for CDI injections
  • org.jboss.weld.environment.servlet.BeanManagerResourceBindingListener binds the reference to the BeanManager to JNDI location java:comp/env/BeanManager to make it accessible anywhere from the application

With that, the full power of CDI 1.1 is at your disposal. Let us introduce the PeopleService class annotated with @Named annotation and with an initialization method declared and annotated with @PostConstruct just to create one person.

@Named
public class PeopleService {
    private final ConcurrentMap< String, Person > persons = 
        new ConcurrentHashMap< String, Person >(); 
 
    @PostConstruct
    public void init() {  
        persons.put( "a@b.com", new Person( "a@b.com", "Tom", "Bombadilt" ) );
    }
    
    // Additional methods 
    // ...
}   

Up to now we have said nothing about configuring JAX-RS 2.0 applications and resources in CDI 1.1 enviroment. The reason for that is very simple: depending on the application, you may go with zero-effort configuration or fully customizable one. Let us go through both approaches.

With zero-effort configuration, you may define an empty JAX-RS 2.0 application and any number of JAX-RS 2.0 resources: Apache CXF 3.0 implicitly will wire them together by associating each resource class with this application. Here is an example of JAX-RS 2.0 application:

package com.example.rs;

import javax.ws.rs.ApplicationPath;
import javax.ws.rs.core.Application;

@ApplicationPath( "api" )
public class JaxRsApiApplication extends Application {
}

And here is a JAX-RS 2.0 resource PeopleRestService which injects the PeopleService managed bean:

package com.example.rs;

import java.util.Collection;

import javax.inject.Inject;
import javax.ws.rs.DELETE;
import javax.ws.rs.DefaultValue;
import javax.ws.rs.FormParam;
import javax.ws.rs.GET;
import javax.ws.rs.POST;
import javax.ws.rs.PUT;
import javax.ws.rs.Path;
import javax.ws.rs.PathParam;
import javax.ws.rs.Produces;
import javax.ws.rs.QueryParam;
import javax.ws.rs.core.Context;
import javax.ws.rs.core.MediaType;
import javax.ws.rs.core.Response;
import javax.ws.rs.core.UriInfo;

import com.example.model.Person;
import com.example.services.PeopleService;

@Path( "/people" )
public class PeopleRestService {
    @Inject private PeopleService peopleService;
 
    @Produces( { MediaType.APPLICATION_JSON } )
    @GET
    public Collection< Person > getPeople( @QueryParam( "page") @DefaultValue( "1" ) final int page ) {
        // ...
    }

    @Produces( { MediaType.APPLICATION_JSON } )
    @Path( "/{email}" )
    @GET
    public Person getPerson( @PathParam( "email" ) final String email ) {
        // ...
    }

    @Produces( { MediaType.APPLICATION_JSON  } )
    @POST
    public Response addPerson( @Context final UriInfo uriInfo,
            @FormParam( "email" ) final String email, 
            @FormParam( "firstName" ) final String firstName, 
            @FormParam( "lastName" ) final String lastName ) {
        // ...
    }
 
    // More HTTP methods here 
    // ...
}
Nothing else is required: Apache CXF 3.0 application could be run like that and be fully functional. The complete source code of the sample project is available on GitHub. Please keep in mind that if you are following this style, only single empty JAX-RS 2.0 application should be declared.

With customizable approach more options are available but a bit more work have to be done. Each JAX-RS 2.0 application should provide non-empty getClasses() or/and getSingletons() collections implementation. However, JAX-RS 2.0 resource classes stay unchanged. Here is an example (which basically leads to the same application configuration we have seen before):

package com.example.rs;

import java.util.Arrays;
import java.util.HashSet;
import java.util.Set;

import javax.enterprise.inject.Produces;
import javax.inject.Inject;
import javax.ws.rs.ApplicationPath;
import javax.ws.rs.core.Application;

import com.fasterxml.jackson.jaxrs.json.JacksonJsonProvider;

@ApplicationPath( "api" )
public class JaxRsApiApplication extends Application {
    @Inject private PeopleRestService peopleRestService;
    @Produces private JacksonJsonProvider jacksonJsonProvider = new JacksonJsonProvider();  
 
    @Override
    public Set< Object > getSingletons() {
        return new HashSet<>(
            Arrays.asList( 
                peopleRestService, 
                jacksonJsonProvider 
            )
        );
    }
}
Please notice, that JAXRSCdiResourceExtension portable CDI 1.1 extension automatically creates managed beans for each JAX-RS 2.0 applications (the ones extending Application) and resources (annotated with @Path). As such, those are immediately available for injection (as for example PeopleRestService in the snippet above). The class JacksonJsonProvider is annotated with @Provider annotation and as such will be treated as JAX-RS 2.0 provider. There are no limit on JAX-RS 2.0 applications which could be defined in this way. The complete source code of the sample project using this appoarch is available on GitHub

No matter which approach you have chosen, our sample application is going to work the same. Let us build it and run:

> mvn clean package
> java -jar target/jax-rs-2.0-cdi-0.0.1-SNAPSHOT.jar 

Calling the couple of implemented REST APIs confirms that application is functioning and configured properly. Let us issue the GET command to ensure that the method of PeopleService annotated with @PostConstruct has been called upon managed bean creation.

> curl http://localhost:8080/rest/api/people

HTTP/1.1 200 OK
Content-Type: application/json
Date: Thu, 29 May 2014 22:39:35 GMT
Transfer-Encoding: chunked
Server: Jetty(9.1.z-SNAPSHOT)

[{"email":"a@b.com","firstName":"Tom","lastName":"Bombadilt"}]

And here is the example of POST command:

> curl -i http://localhost:8080/rest/api/people -X POST -d "email=a@c.com&firstName=Tom&lastName=Knocker"

HTTP/1.1 201 Created
Content-Type: application/json
Date: Thu, 29 May 2014 22:40:08 GMT
Location: http://localhost:8080/rest/api/people/a@c.com
Transfer-Encoding: chunked
Server: Jetty(9.1.z-SNAPSHOT)

{"email":"a@c.com","firstName":"Tom","lastName":"Knocker"}

In this blog post we have just scratched the surface of what is possible now with Apache CXF and CDI 1.1 integration. Just to mention that embedded Apache Tomcat 7.x / 8.x as well as WAR-based deployments of Apache CXF with CDI 1.1 are possible on most JEE application servers and servlet containers.

Please take a look on official documentation and give it a try!

The complete source code is available on GitHub.

Thursday, March 27, 2014

Apache CXF 3.0: JAX-RS 2.0 and Bean Validation 1.1 finally together

The upcoming release 3.0 (currently in milestone 2 phase) of the great Apache CXF framework is bringing a lot of interesting and useful features, getting closer to deliver full-fledged JAX-RS 2.0 support. One of those features, a long-awaited by many of us, is the support of Bean Validation 1.1: easy and concise model to add validation capabilities to your REST services layer.

In this blog post we are going to look how to configure Bean Validation 1.1 in your Apache CXF projects and discuss some interesting use cases. To keep this post reasonably short and focused, we will not discuss the Bean Validation 1.1 itself but concentrate more on integration with JAX-RS 2.0 resources (some of the bean validation basics we have already covered in the older posts).

At the moment, Hibernate Validator is the de-facto reference implementation of Bean Validation 1.1 specification, with the latest version being 5.1.0.Final and as such it will be the validation provider of our choice (Apache BVal project at the moment supports only Bean Validation 1.0). It is worth to mention that Apache CXF is agnostic to implementation and will work equally well either with Hibernate Validator or Apache BVal once released.

We are going to build a very simple application to manage people. Our model consists of one single class named Person.

package com.example.model;

import javax.validation.constraints.NotNull;

import org.hibernate.validator.constraints.Email;

public class Person {
    @NotNull @Email private String email;
    @NotNull private String firstName;
    @NotNull private String lastName;
        
    public Person() {
    }

    public Person( final String email ) {
        this.email = email;
    }

    public String getEmail() {
        return email;
    }

    public void setEmail( final String email ) {
        this.email = email;
    }
    
    public String getFirstName() {
        return firstName;
    }

    public String getLastName() {
        return lastName;
    }

    public void setFirstName( final String firstName ) {
        this.firstName = firstName;
    }
    
    public void setLastName( final String lastName ) {
        this.lastName = lastName;
    }        
}

From snippet above we can see that Person class imposes couple of restrictions on its properties: all of them should not be null. Additionally, email property should contain a valid e-mail address (which will be validated by Hibernate Validator-specific constraint @Email). Pretty simple.

Now, let us take a look on JAX-RS 2.0 resources with validation constraints. The skeleton of the PeopleRestService class is bind to /people URL path and is shown below.

package com.example.rs;

import java.util.Collection;

import javax.inject.Inject;
import javax.validation.Valid;
import javax.validation.constraints.Min;
import javax.validation.constraints.NotNull;
import javax.ws.rs.DELETE;
import javax.ws.rs.DefaultValue;
import javax.ws.rs.FormParam;
import javax.ws.rs.GET;
import javax.ws.rs.POST;
import javax.ws.rs.Path;
import javax.ws.rs.PathParam;
import javax.ws.rs.Produces;
import javax.ws.rs.QueryParam;
import javax.ws.rs.core.Context;
import javax.ws.rs.core.MediaType;
import javax.ws.rs.core.Response;
import javax.ws.rs.core.UriInfo;

import org.hibernate.validator.constraints.Length;

import com.example.model.Person;
import com.example.services.PeopleService;

@Path( "/people" ) 
public class PeopleRestService {
    @Inject private PeopleService peopleService;

    // REST methods here
}

It should look very familiar, nothing new. The first method we are going to add and decorate with validation constraints is getPerson, which will look up a person by its e-mail address.

@Produces( { MediaType.APPLICATION_JSON } )
@Path( "/{email}" )
@GET
public @Valid Person getPerson( 
        @Length( min = 5, max = 255 ) @PathParam( "email" ) final String email ) {
    return peopleService.getByEmail( email );
}

There are a couple of differences from traditional JAX-RS 2.0 method declaration. Firstly, we would like the e-mail address (email path parameter) to be at least 5 characters long (but no more than 255 characters) which is imposed by @Length( min = 5, max = 255 ) annotation. Secondly, we would like to ensure that only valid person is returned by this method so we annotated the method's return value with @Valid annotation. The effect of @Valid is very interesting: the person's instance in question will be checked against all validation constraints declared by its class (Person).

At the moment, Bean Validation 1.1 is not active by default in your Apache CXF projects so if you run your application and call this REST endpoint, all validation constraints will be simply ignored. The good news are that it is very easy to activate Bean Validation 1.1 as it requires only three components to be added to your usual configuration (please check out this feature documentation for more details and advanced configuration):

  • JAXRSBeanValidationInInterceptor in-inteceptor: performs validation of the input parameters of JAX-RS 2.0 resource methods
  • JAXRSBeanValidationOutInterceptor out-inteceptor: performs validation of return values of JAX-RS 2.0 resource methods
  • ValidationExceptionMapper exception mapper: maps the validation violations to HTTP status codes. As per specification, all input parameters validation violations result into 400 Bad Request error. Respectively, all return values validation violations result into 500 Internal Server Error error. At the moment, the ValidationExceptionMapper does not include additional information into response (as it may violate application protocol) but it could be easily extended to provide more details about validation errors.
The AppConfig class shows off one of the ways to wire up all the required components together using RuntimeDelegate and JAXRSServerFactoryBean (the XML-based configuration is also supported).

package com.example.config;

import java.util.Arrays;

import javax.ws.rs.ext.RuntimeDelegate;

import org.apache.cxf.bus.spring.SpringBus;
import org.apache.cxf.endpoint.Server;
import org.apache.cxf.interceptor.Interceptor;
import org.apache.cxf.jaxrs.JAXRSServerFactoryBean;
import org.apache.cxf.jaxrs.validation.JAXRSBeanValidationInInterceptor;
import org.apache.cxf.jaxrs.validation.JAXRSBeanValidationOutInterceptor;
import org.apache.cxf.jaxrs.validation.ValidationExceptionMapper;
import org.apache.cxf.message.Message;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.context.annotation.DependsOn;

import com.example.rs.JaxRsApiApplication;
import com.example.rs.PeopleRestService;
import com.example.services.PeopleService;
import com.fasterxml.jackson.jaxrs.json.JacksonJsonProvider;

@Configuration
public class AppConfig {    
    @Bean( destroyMethod = "shutdown" )
    public SpringBus cxf() {
        return new SpringBus();
    }

    @Bean @DependsOn( "cxf" )
    public Server jaxRsServer() {
        final JAXRSServerFactoryBean factory = 
            RuntimeDelegate.getInstance().createEndpoint( 
                jaxRsApiApplication(), 
                JAXRSServerFactoryBean.class 
            );
        factory.setServiceBeans( Arrays.< Object >asList( peopleRestService() ) );
        factory.setAddress( factory.getAddress() );
        factory.setInInterceptors( 
            Arrays.< Interceptor< ? extends Message > >asList( 
                new JAXRSBeanValidationInInterceptor()
            ) 
        );
        factory.setOutInterceptors( 
            Arrays.< Interceptor< ? extends Message > >asList( 
                new JAXRSBeanValidationOutInterceptor() 
            ) 
        );
        factory.setProviders( 
            Arrays.asList( 
                new ValidationExceptionMapper(), 
                new JacksonJsonProvider() 
            ) 
        );

        return factory.create();
    }
    
    @Bean 
    public JaxRsApiApplication jaxRsApiApplication() {
        return new JaxRsApiApplication();
    }
    
    @Bean 
    public PeopleRestService peopleRestService() {
        return new PeopleRestService();
    }

    @Bean 
    public PeopleService peopleService() {
        return new PeopleService();
    }
}

All in/out interceptors and exception mapper are injected. Great, let us build the project and run the server to validate the Bean Validation 1.1 is active and works as expected.

mvn clean package
java -jar target/jaxrs-2.0-validation-0.0.1-SNAPSHOT.jar 

Now, if we issue a REST request with short (or invalid) e-mail address a@b, the server should return 400 Bad Request. Let us validate that.

> curl http://localhost:8080/rest/api/people/a@b -i

HTTP/1.1 400 Bad Request
Date: Wed, 26 Mar 2014 00:11:59 GMT
Content-Length: 0
Server: Jetty(9.1.z-SNAPSHOT)

Excellent! To be completely sure, we can check server console output and find there the validation exception of type ConstraintViolationException and its stacktrace. Plus, the last line provides the details what went wrong: PeopleRestService.getPerson.arg0: length must be between 5 and 255 (please notice, because argument names are not currently available on JVM after compilation, they are replaced by placeholders like arg0, arg1, ...).

WARNING: Interceptor for {http://rs.example.com/}PeopleRestService has thrown exception, unwinding now
javax.validation.ConstraintViolationException
        at org.apache.cxf.validation.BeanValidationProvider.validateParameters(BeanValidationProvider.java:119)
        at org.apache.cxf.validation.BeanValidationInInterceptor.handleValidation(BeanValidationInInterceptor.java:59)
        at org.apache.cxf.validation.AbstractValidationInterceptor.handleMessage(AbstractValidationInterceptor.java:73)
        at org.apache.cxf.phase.PhaseInterceptorChain.doIntercept(PhaseInterceptorChain.java:307)
        at org.apache.cxf.transport.ChainInitiationObserver.onMessage(ChainInitiationObserver.java:121)
        at org.apache.cxf.transport.http.AbstractHTTPDestination.invoke(AbstractHTTPDestination.java:240)
        at org.apache.cxf.transport.servlet.ServletController.invokeDestination(ServletController.java:223)
        at org.apache.cxf.transport.servlet.ServletController.invoke(ServletController.java:197)
        at org.apache.cxf.transport.servlet.ServletController.invoke(ServletController.java:149)
        at org.apache.cxf.transport.servlet.CXFNonSpringServlet.invoke(CXFNonSpringServlet.java:167)
        at org.apache.cxf.transport.servlet.AbstractHTTPServlet.handleRequest(AbstractHTTPServlet.java:286)
        at org.apache.cxf.transport.servlet.AbstractHTTPServlet.doGet(AbstractHTTPServlet.java:211)
        at javax.servlet.http.HttpServlet.service(HttpServlet.java:687)
        at org.apache.cxf.transport.servlet.AbstractHTTPServlet.service(AbstractHTTPServlet.java:262)
        at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:711)
        at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:552)
        at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1112)
        at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:479)
        at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1046)
        at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
        at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97)
        at org.eclipse.jetty.server.Server.handle(Server.java:462)
        at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:281)
        at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:232)
        at org.eclipse.jetty.io.AbstractConnection$1.run(AbstractConnection.java:505)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:607)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:536)
        at java.lang.Thread.run(Unknown Source)

Mar 25, 2014 8:11:59 PM org.apache.cxf.jaxrs.validation.ValidationExceptionMapper toResponse
WARNING: PeopleRestService.getPerson.arg0: length must be between 5 and 255

Moving on, we are going to add two more REST methods to demonstrate the collections and Response validation in action.

@Produces( { MediaType.APPLICATION_JSON } )
@GET
public @Valid Collection< Person > getPeople( 
        @Min( 1 ) @QueryParam( "count" ) @DefaultValue( "1" ) final int count ) {
    return peopleService.getPeople( count );
}

The @Valid annotation on collection of objects will ensure that every single object in collection is valid. The count parameter is also constrained to have the minimum value 1 by @Min( 1 ) annotation (the @DefaultValue is taken into account if the query parameter is not specified). Let us on purpose add the person without first and last names set so the resulting collection will contain at least one person instance which should not pass the validation process.

> curl http://localhost:8080/rest/api/people -X POST -id "email=a@b3.com"

With that, the call of getPeople REST method should return 500 Internal Server Error. Let us check that is the case.

> curl -i http://localhost:8080/rest/api/people?count=10

HTTP/1.1 500 Server Error
Date: Wed, 26 Mar 2014 01:28:58 GMT
Content-Length: 0
Server: Jetty(9.1.z-SNAPSHOT)

Looking into server console output, the hint what is wrong is right there.

Mar 25, 2014 9:28:58 PM org.apache.cxf.jaxrs.validation.ValidationExceptionMapper toResponse
WARNING: PeopleRestService.getPeople.[0].firstName: may not be null
Mar 25, 2014 9:28:58 PM org.apache.cxf.jaxrs.validation.ValidationExceptionMapper toResponse
WARNING: PeopleRestService.getPeople.[0].lastName: may not be null

And finally, yet another example, this time with generic Response object.

@Valid
@Produces( { MediaType.APPLICATION_JSON  } )
@POST
public Response addPerson( @Context final UriInfo uriInfo,
        @NotNull @Length( min = 5, max = 255 ) @FormParam( "email" ) final String email, 
        @FormParam( "firstName" ) final String firstName, 
        @FormParam( "lastName" ) final String lastName ) {        
    final Person person = peopleService.addPerson( email, firstName, lastName );
    return Response.created( uriInfo.getRequestUriBuilder().path( email ).build() )
        .entity( person ).build();
}

The last example is a bit tricky: the Response class is part of JAX-RS 2.0 API and has no validation constraints defined. As such, imposing any validation rules on the instance of this class will not trigger any violations. But Apache CXF tries its best and performs a simple but useful trick: instead of Response instance, the response's entity will be validated instead. We can easy verify that by trying to create a person without first and last names set: the expected result should be 500 Internal Server Error.

> curl http://localhost:8080/rest/api/people -X POST -id "email=a@b3.com"

HTTP/1.1 500 Server Error
Date: Wed, 26 Mar 2014 01:13:06 GMT
Content-Length: 0
Server: Jetty(9.1.z-SNAPSHOT)

And server console output is more verbose:

Mar 25, 2014 9:13:06 PM org.apache.cxf.jaxrs.validation.ValidationExceptionMapper toResponse
WARNING: PeopleRestService.addPerson.<return value>.firstName: may not be null
Mar 25, 2014 9:13:06 PM org.apache.cxf.jaxrs.validation.ValidationExceptionMapper toResponse
WARNING: PeopleRestService.addPerson.<return value>.lastName: may not be null

Nice! In this blog post we have just touched a bit the a topic of how Bean Validation 1.1 may make your Apache CXF projects better by providing such a rich and extensible declarative validation support. Definitely give it a try!

A complete project is available on GitHub.

Sunday, February 23, 2014

Knowing how all your components work together: distributed tracing with Zipkin

In today's post we will try to cover very interesting and important topic: distributed system tracing. What it practically means is that we will try to trace the request from the point it was issued by the client to the point the response to this request was received. At first, it looks quite straightforward but in reality it may involve many calls to several other systems, databases, NoSQL stores, caches, you name it ...

In 2010 Google published a paper about Dapper, a large-scale distributed systems tracing infrastructure (very interesting reading by the way). Later on, Twitter built its own implementation based on Dapper paper, called Zipkin and that's the one we are going to look at.

We will build a simple JAX-RS 2.0 server using great Apache CXF library. For the client side, we will use JAX-RS 2.0 client API and by utilizing Zipkin we will trace all the interactions between the client and the server (as well as everything happening on server side). To make an example a bit more illustrative, we will pretend that server uses some kind of database to retrieve the data. Our code will be a mix of pure Java and a bit of Scala (the choice of Scala will be cleared up soon).

One additional dependency in order for Zipkin to work is Apache Zookeeper. It is required for coordination and should be started in advance. Luckily, it is very easy to do:

  • download the release from http://zookeeper.apache.org/releases.html (the current stable version at the moment of writing is 3.4.5)
  • unpack it into zookeeper-3.4.5
  • copy zookeeper-3.4.5/conf/zoo_sample.cfg to zookeeper-3.4.5/conf/zoo.cfg
  • and just start Apache Zookeeper server
    Windows: zookeeper-3.4.5/bin/zkServer.cmd
    Linux: zookeeper-3.4.5/bin/zkServer.sh start

Now back to Zipkin. Zipkin is written in Scala. It is still in active development and the best way to start off with it is just by cloning its GitHub repository and build it from sources:

git clone https://github.com/twitter/zipkin.git

From architectural prospective, Zipkin consists of three main components:

  • collector: collects traces across the system
  • query: queries collected traces
  • web: provides web-based UI to show the traces

To run them, Zipkin guys provide useful scripts in the bin folder with the only requirement that JDK 1.7 should be installed:

  • bin/collector
  • bin/query
  • bin/web
Let's execute these scripts and ensure that every component has been started successfully, with no stack traces on the console (for curious readers, I was not able to make Zipkin work on Windows so I assume we are running it on Linux box). By default, Zipkin web UI is available on port 8080. The storage for traces is embedded SQLite engine. Though it works, the better storages (like awesome Redis) are available.

The preparation is over, let's do some code. We will start with JAX-RS 2.0 client part as it's very straightforward (ClientStarter.java):

package com.example.client;

import javax.ws.rs.client.Client;
import javax.ws.rs.client.ClientBuilder;
import javax.ws.rs.core.MediaType;
import javax.ws.rs.core.Response;

import com.example.zipkin.Zipkin;
import com.example.zipkin.client.ZipkinRequestFilter;
import com.example.zipkin.client.ZipkinResponseFilter;

public class ClientStarter {
  public static void main( final String[] args ) throws Exception { 
    final Client client = ClientBuilder
      .newClient()
      .register( new ZipkinRequestFilter( "People", Zipkin.tracer() ), 1 )
      .register( new ZipkinResponseFilter( "People", Zipkin.tracer() ), 1 );        
                        
    final Response response = client
      .target( "http://localhost:8080/rest/api/people" )
      .request( MediaType.APPLICATION_JSON )
      .get();

    if( response.getStatus() == 200 ) {
      System.out.println( response.readEntity( String.class ) );
    }
        
    response.close();
    client.close();
        
    // Small delay to allow tracer to send the trace over the wire
    Thread.sleep( 1000 );
  }
}

Except a couple of imports and classes with Zipkin in it, everything should look simple. So what those ZipkinRequestFilter and ZipkinResponseFilter are for? Zipkin is awesome but it's not a magical tool. In order to trace any request in distributed system, there should be some context passed along with it. In REST/HTTP world, it's usually request/response headers. Let's take a look on ZipkinRequestFilter first (ZipkinRequestFilter.scala):

package com.example.zipkin.client

import javax.ws.rs.client.ClientRequestFilter
import javax.ws.rs.ext.Provider
import javax.ws.rs.client.ClientRequestContext
import com.twitter.finagle.http.HttpTracing
import com.twitter.finagle.tracing.Trace
import com.twitter.finagle.tracing.Annotation
import com.twitter.finagle.tracing.TraceId
import com.twitter.finagle.tracing.Tracer

@Provider
class ZipkinRequestFilter( val name: String, val tracer: Tracer ) extends ClientRequestFilter {
  def filter( requestContext: ClientRequestContext ): Unit = {      
    Trace.pushTracerAndSetNextId( tracer, true )
 
    requestContext.getHeaders().add( HttpTracing.Header.TraceId, Trace.id.traceId.toString )
    requestContext.getHeaders().add( HttpTracing.Header.SpanId, Trace.id.spanId.toString )
         
    Trace.id._parentId foreach { id => 
      requestContext.getHeaders().add( HttpTracing.Header.ParentSpanId, id.toString ) 
    }    

    Trace.id.sampled foreach { sampled => 
      requestContext.getHeaders().add( HttpTracing.Header.Sampled, sampled.toString ) 
    }

    requestContext.getHeaders().add( HttpTracing.Header.Flags, Trace.id.flags.toLong.toString )
             
    if( Trace.isActivelyTracing ) {
      Trace.recordRpcname( name,  requestContext.getMethod() )
      Trace.recordBinary( "http.uri", requestContext.getUri().toString()  )
      Trace.record( Annotation.ClientSend() )    
    }
  }
}

A bit of Zipkin internals will make this code superclear. The central part of Zipkin API is Trace class. Every time we would like to initiate tracing, we should have a Trace Id and the tracer to actually trace it. This single line generates new Trace Id and register the tracer (internally this data is held in thread local state).

Trace.pushTracerAndSetNextId( tracer, true )

Traces are hierarchical by nature, so do Trace Ids: every Trace Id could be a root or part of another trace. In our example, we know for sure that we are the first and as such the root of the trace. Later on the Trace Id is wrapped into HTTP headers and will be passed along the request (we will see on server side how it is being used). The last three lines associate the useful information with the trace: name of our API (People), HTTP method, URI and most importantly, that it's the client sending the request to the server.

Trace.recordRpcname( name,  requestContext.getMethod() )
Trace.recordBinary( "http.uri", requestContext.getUri().toString()  )
Trace.record( Annotation.ClientSend() ) 

The ZipkinResponseFilter does the reverse to ZipkinRequestFilter and extract the Trace Id from the request headers (ZipkinResponseFilter.scala):

package com.example.zipkin.client

import javax.ws.rs.client.ClientResponseFilter
import javax.ws.rs.client.ClientRequestContext
import javax.ws.rs.client.ClientResponseContext
import javax.ws.rs.ext.Provider
import com.twitter.finagle.tracing.Trace
import com.twitter.finagle.tracing.Annotation
import com.twitter.finagle.tracing.SpanId
import com.twitter.finagle.http.HttpTracing
import com.twitter.finagle.tracing.TraceId
import com.twitter.finagle.tracing.Flags
import com.twitter.finagle.tracing.Tracer

@Provider
class ZipkinResponseFilter( val name: String, val tracer: Tracer ) extends ClientResponseFilter {  
  def filter( requestContext: ClientRequestContext, responseContext: ClientResponseContext ): Unit = {
    val spanId = SpanId.fromString( requestContext.getHeaders().getFirst( HttpTracing.Header.SpanId ).toString() )

    spanId foreach { sid =>
      val traceId = SpanId.fromString( requestContext.getHeaders().getFirst( HttpTracing.Header.TraceId ).toString() )
        
      val parentSpanId = requestContext.getHeaders().getFirst( HttpTracing.Header.ParentSpanId ) match {
        case s: String => SpanId.fromString( s.toString() )
        case _ => None
      }

      val sampled = requestContext.getHeaders().getFirst( HttpTracing.Header.Sampled ) match { 
        case s: String =>  s.toString.toBoolean
        case _ => true
      }
        
      val flags = Flags( requestContext.getHeaders().getFirst( HttpTracing.Header.Flags ).toString.toLong )        
      Trace.setId( TraceId( traceId, parentSpanId, sid, Option( sampled ), flags ) )
    }
      
    if( Trace.isActivelyTracing ) {
      Trace.record( Annotation.ClientRecv() )
    }
  }
}

Strictly speaking, in our example it's not necessary to extract the Trace Id from the request because both filters should be executed by the single thread. But the last line is very important: it marks the end of our trace by saying that client has received the response.

Trace.record( Annotation.ClientRecv() )

What's left is actually the tracer itself (Zipkin.scala):

package com.example.zipkin

import com.twitter.finagle.stats.DefaultStatsReceiver
import com.twitter.finagle.zipkin.thrift.ZipkinTracer
import com.twitter.finagle.tracing.Trace
import javax.ws.rs.ext.Provider

object Zipkin {
  lazy val tracer = ZipkinTracer.mk( host = "localhost", port = 9410, DefaultStatsReceiver, 1 )
}

If at this point you are confused what all those traces and spans mean please look through this documentation page, you will get the basic understanding of those concepts.

At this point, there is nothing left on the client side and we are good to move to the server side. Our JAX-RS 2.0 server will expose the single endpoint (PeopleRestService.java):

package com.example.server.rs;

import java.util.Arrays;
import java.util.Collection;
import java.util.concurrent.Callable;

import javax.ws.rs.GET;
import javax.ws.rs.Path;
import javax.ws.rs.Produces;

import com.example.model.Person;
import com.example.zipkin.Zipkin;

@Path( "/people" ) 
public class PeopleRestService {
  @Produces( { "application/json" } )
  @GET
  public Collection< Person > getPeople() {
    return Zipkin.invoke( "DB", "FIND ALL", new Callable< Collection< Person > >() {
      @Override
      public Collection call() throws Exception {
        return Arrays.asList( new Person( "Tom", "Bombdil" ) );
      }   
    } );   
  }
}

As we mentioned before, we will simulate the access to database and generate a child trace by using Zipkin.invoke wrapper (which looks very simple, Zipkin.scala):

package com.example.zipkin

import java.util.concurrent.Callable
import com.twitter.finagle.stats.DefaultStatsReceiver
import com.twitter.finagle.tracing.Trace
import com.twitter.finagle.zipkin.thrift.ZipkinTracer
import com.twitter.finagle.tracing.Annotation

object Zipkin {
  lazy val tracer = ZipkinTracer.mk( host = "localhost", port = 9410, DefaultStatsReceiver, 1 )
    
  def invoke[ R ]( service: String, method: String, callable: Callable[ R ] ): R = Trace.unwind {
    Trace.pushTracerAndSetNextId( tracer, false )      
      
    Trace.recordRpcname( service, method );
    Trace.record( new Annotation.ClientSend() );
          
    try {
      callable.call()
    } finally {
      Trace.record( new Annotation.ClientRecv() );
    }
  }   
}

As we can see, in this case the server itself becomes a client for some other service (database).

The last and most important part of the server is to intercept all HTTP requests, extract the Trace Id from them so it will be possible to associate more data with the trace (annotate the trace). In Apache CXF it's very easy to do by providing own invoker (ZipkinTracingInvoker.scala):

package com.example.zipkin.server

import org.apache.cxf.jaxrs.JAXRSInvoker
import com.twitter.finagle.tracing.TraceId
import org.apache.cxf.message.Exchange
import com.twitter.finagle.tracing.Trace
import com.twitter.finagle.tracing.Annotation
import org.apache.cxf.jaxrs.model.OperationResourceInfo
import org.apache.cxf.jaxrs.ext.MessageContextImpl
import com.twitter.finagle.tracing.SpanId
import com.twitter.finagle.http.HttpTracing
import com.twitter.finagle.tracing.Flags
import scala.collection.JavaConversions._
import com.twitter.finagle.tracing.Tracer
import javax.inject.Inject

class ZipkinTracingInvoker extends JAXRSInvoker {
  @Inject val tracer: Tracer = null
  
  def trace[ R ]( exchange: Exchange )( block: => R ): R = {
    val context = new MessageContextImpl( exchange.getInMessage() )
    Trace.pushTracer( tracer )
        
    val id = Option( exchange.get( classOf[ OperationResourceInfo ] ) ) map { ori =>
      context.getHttpHeaders().getRequestHeader( HttpTracing.Header.SpanId ).toList match {
        case x :: xs => SpanId.fromString( x ) map { sid => 
          val traceId = context.getHttpHeaders().getRequestHeader( HttpTracing.Header.TraceId ).toList match {
            case x :: xs => SpanId.fromString( x )
            case _ => None
          }
          
          val parentSpanId = context.getHttpHeaders().getRequestHeader( HttpTracing.Header.ParentSpanId ).toList match {
            case x :: xs => SpanId.fromString( x )
            case _ => None
          }
  
          val sampled = context.getHttpHeaders().getRequestHeader( HttpTracing.Header.Sampled ).toList match { 
            case x :: xs =>  x.toBoolean
            case _ => true
          }
                    
          val flags = context.getHttpHeaders().getRequestHeader( HttpTracing.Header.Flags ).toList match {
            case x :: xs =>  Flags( x.toLong )
            case _ => Flags()
          }
         
          val id = TraceId( traceId, parentSpanId, sid, Option( sampled ), flags )                     
          Trace.setId( id )
        
          if( Trace.isActivelyTracing ) {
            Trace.recordRpcname( context.getHttpServletRequest().getProtocol(), ori.getHttpMethod() )
            Trace.record( Annotation.ServerRecv() )
          }
        
          id
        }           
          
        case _ => None
      }
    }
    
    val result = block
    
    if( Trace.isActivelyTracing ) {
      id map { id => Trace.record( new Annotation.ServerSend() ) }
    }
    
    result
  }
  
  @Override
  override def invoke( exchange: Exchange, parametersList: AnyRef ): AnyRef = {
    trace( exchange )( super.invoke( exchange, parametersList ) )     
  }
}

Basically, the only thing this code does is extracting Trace Id from request and associating it with the current thread. Also please notice that we associate additional data with the trace marking the server participation.

  Trace.recordRpcname( context.getHttpServletRequest().getProtocol(), ori.getHttpMethod() )
  Trace.record( Annotation.ServerRecv() )

To see the tracing in live, let's start our server (please notice that sbt should be installed), assuming all Zipkin components and Apache Zookeeper are already up and running:

sbt 'project server' 'run-main com.example.server.ServerStarter'
then the client:
sbt 'project client' 'run-main com.example.client.ClientStarter'
and finally open Zipkin web UI at http://localhost:8080. We should see something like that (depending how many times you have run the client):

Alternatively, we can build and run fat JARs using sbt-assembly plugin:

sbt assembly
java -jar server/target/zipkin-jaxrs-2.0-server-assembly-0.0.1-SNAPSHOT.jar 
java -jar client/target/zipkin-jaxrs-2.0-client-assembly-0.0.1-SNAPSHOT.jar 

If we click on any particular trace, the more detailed information will be shown, much resembling client <-> server <-> database chain.

Even more details are shown when we click on particular element in the tree.

Lastly, the bonus part is components / services dependency graph.

As we can see, all the data associated with the trace is here and follows hierarchical structure. The root and child traces are detected and shown, as well as timelines for client send/receive and server receive/send chains. Our example is quite naive and simple, but even like that it demonstrates how powerful and useful distributed system tracing is. Thanks to Zipkin guys.

The complete source code is available on GitHub.

Monday, January 13, 2014

Your build tool is your good friend: what sbt can do for Java developer

I think for developers picking the right build tool is a very important choice. For years I have been sticking to Apache Maven and, honestly, it does the job well enough, even nowadays it's a good tool to use. But I always feel it could be done much better ... and then Gradle came along ...

Despite many hours I have spent getting accustomed to Gradle way to do things, I finally gave up and switched back to Apache Maven. The reason - I didn't feel comfortable with it, mostly because of Groovy DSL. Anyway, I think Gradle is great, powerful and extensible build tool which is able to perform any task your build process needs.

But engaging myself more and more with Scala, I quickly discovered sbt. Though sbt is acronym for "simple build tool", my first impression was quite a contrary: I found it complicated and hard to understand. For some reasons, I liked it nonetheless and by spending more time reading the documentation (which is getting better and better), many experiments, I finally would say the choice is made. In this post I would like to show up couple of great things sbt can do to make Java developer's life easy (some knowledge of Scala would be very handy, but it's not required).

Before moving on to real example, couple of facts about sbt. It uses Scala as a language for build scenario and requires a launcher which could be downloaded from here (the version we'll be using is 0.13.1). There are several ways to describe build in sbt, the one this post demonstrates is by using Build.scala with single project.

Our example is a simple Spring console application with couple of JUnit test cases: just enough to see how build with external dependencies is structured and tests are run. Application contains only two classes:

package com.example;

import org.springframework.stereotype.Service;

@Service
public class SimpleService {
    public String getResult() {
        return "Result";
    }
}
and
package com.example;

import org.springframework.context.annotation.AnnotationConfigApplicationContext;
import org.springframework.context.annotation.ComponentScan;
import org.springframework.context.annotation.Configuration;
import org.springframework.context.support.GenericApplicationContext;

public class Starter {
    @Configuration
    @ComponentScan( basePackageClasses = SimpleService.class )
    public static class AppConfig {  
    }
 
    public static void main( String[] args ) {
        try( GenericApplicationContext context = new AnnotationConfigApplicationContext( AppConfig.class ) ) {
            final SimpleService service = context.getBean( SimpleService.class );
            System.out.println( service.getResult() );
        }
    }
}

Now, let see how sbt build looks like. By convention, Build.scala should be located in project subfolder. Additionally, there should be present build.properties file with desired sbt version and plugins.sbt with external plugins (we will use sbteclipse plugin to generate Eclipse project files). We will start with build.properties which contains only one line:

sbt.version=0.13.1

and continue with plugins.sbt, which in our case is also just one line:

addSbtPlugin("com.typesafe.sbteclipse" % "sbteclipse-plugin" % "2.4.0")

Lastly, let's start with the heart of our build: Build.scala. There would be two parts in it: common settings for all projects in our build (useful for multi-project builds but we have only one now) and here is the snippet of this part:

import sbt._
import Keys._
import com.typesafe.sbteclipse.core.EclipsePlugin._

object ProjectBuild extends Build {
  override val settings = super.settings ++ Seq(
    organization := "com.example",    
    name := "sbt-java",    
    version := "0.0.1-SNAPSHOT",    
    
    scalaVersion := "2.10.3",
    scalacOptions ++= Seq( "-encoding", "UTF-8", "-target:jvm-1.7" ),    
    javacOptions ++= Seq( "-encoding", "UTF-8", "-source", "1.7", "-target", "1.7" ),        
    outputStrategy := Some( StdoutOutput ),
    compileOrder := CompileOrder.JavaThenScala,
    
    resolvers ++= Seq( 
      Resolver.mavenLocal, 
      Resolver.sonatypeRepo( "releases" ), 
      Resolver.typesafeRepo( "releases" )
    ),        

    crossPaths := false,            
    fork in run := true,
    connectInput in run := true,
    
    EclipseKeys.executionEnvironment := Some(EclipseExecutionEnvironment.JavaSE17)
  )
}

The build above looks quite clean and understandable: resolvers is a straight analogy of Apache Maven repositories, EclipseKeys.executionEnvironment is customization for execution environment (Java SE 7) for generated Eclipse project. All these keys are very well documented.

Second part is much smaller and defines our main project in terms of dependencies and main class:

lazy val main = Project( 
  id = "sbt-java",  
  base = file("."), 
  settings = Project.defaultSettings ++ Seq(              
    mainClass := Some( "com.example.Starter" ),
     
    initialCommands in console += """
      import com.example._
      import com.example.Starter._
      import org.springframework.context.annotation._
    """,
       
    libraryDependencies ++= Seq(
      "org.springframework" % "spring-context" % "4.0.0.RELEASE",
      "org.springframework" % "spring-beans" % "4.0.0.RELEASE",
      "org.springframework" % "spring-test" % "4.0.0.RELEASE" % "test",
      "com.novocode" % "junit-interface" % "0.10" % "test",
      "junit" % "junit" % "4.11" % "test"
    )
  ) 
)

The initialCommands requires a bit of explanation here: sbt is able to run Scala console (REPL) and this setting allows to add default import statements so we can use our classes immediately. The dependency to junit-interface allows sbt to run JUnit test cases and it's the first thing we'll do: add some tests. Before creating actual tests, we will start sbt and ask it to run test cases on every code change, just like that:

sbt ~test

While sbt is running, we will add a test case:

package com.example;

import static org.hamcrest.core.IsEqual.equalTo;
import static org.junit.Assert.assertThat;

import org.junit.After;
import org.junit.Before;
import org.junit.Test;
import org.springframework.context.annotation.AnnotationConfigApplicationContext;
import org.springframework.context.support.GenericApplicationContext;

import com.example.Starter.AppConfig;

public class SimpleServiceTestCase {
    private GenericApplicationContext context;
    private SimpleService service;
 
    @Before
    public void setUp() {
        context = new AnnotationConfigApplicationContext( AppConfig.class );
        service = context.getBean( SimpleService.class );
    }
 
    @After
    public void tearDown() {
        context.close();
    }

    @Test
    public void testSampleTest() { 
        assertThat( service.getResult(), equalTo( "Result" ) );
    } 
}

In a console we should see that sbt picked the change automatically and run all test cases. Sadly, because of this issue which is already fixed and should be available in next release of junit-interface, we cannot use @RunWith and @ContextConfiguration annotation to run Spring test cases yet.

For TDD practitioners it's an awesome feature to have. The next terrific feature we are going to look at is Scala console (RELP) which gives as the ability to play with application without actually running it. It could be invoked by typing:

sbt console

and observing something like this in the terminal (as we can see, the imports from initialCommands are automatically included):

At this moment playground is established and we can do a lot of very interesting things, for example: create context, get beans and call any methods on them:

sbt takes care about classpath so all your classes and external dependencies are available for use. I found this way to discover things much faster than by using debugger or other techniques.

At the moment, there is no good support for sbt in Eclipse but it's very easy to generate Eclipse project files by using sbteclipse plugin we've touched before:

sbt eclipse

Awesome! Not to mention other great plugins which are kindly listed here and the ability to import Apache Maven POM files using externalPom() which really simplifies the migration. As a conclusion from my side, if you are looking for better, modern, extensible build tool for your project, please take a look at sbt. It's a great piece of software built on top of awesome, consise language.

Complete project is available on GitHub.