english

56 posts

New Book - A Practical Guide for Java 8 Lambdas and Streams

This book is not the first book about Java 8 lambda expressions and streams, and it's definitely not the last book about lambda expressions and streams. Java 8 is a Java platform upgrade which the community looking forward to for a long time. Lambda expressions and streams quickly gain popularity in Java developers. There are already a lot of books and online tutorials about lambda expressions and streams. This book is trying to explain lambda expressions and streams from a different perspective.

  • For lambda expressions, this book explains in details based on JSR 335.
  • For streams, this book covers fundamental concepts of Java core library.
  • This book provides how-to examples for lambda expressions and streams.
  • This book also covers the important utility class Optional.

Lambda expressions and streams are easy to understand and use. This book tries to provide some insights about how to use them efficiently.

Buy this book

New Book - Build Mobile Apps with Ionic and Firebase

With the prevalence of mobile apps, more and more developers want to learn how to build mobile apps. Developers can choose iOS or Android platforms to develop mobile apps. But learning Objective-C/Swift or Java is not an easy task. Web development programming languages, HTML, JavaScript and CSS, are easier to understand and learn. Building mobile apps is made possible by Apache Cordova, which creates a new type of mobile apps - Hybrid mobile apps. Hybrid mobile apps are actually running in an internal brower inside a wrapper created by Apache Cordova. With hybrid mobile apps, we can have one single code base for different platforms. Developers also can use their existing web development skills.

Ionic framework builds on top of Apache Cordova and provides out-of-box components which make developing hybrid mobile apps much easier. Ionic uses Angular as the JavaScript framework and has nice default UI style with similar look & feel as native apps. Firebase is a realtime database which can be accessed in web apps using JavaScript. With Ionic and Firebase, you just need to develop front-end code. You don't need to manage any back-end code or servers.

This book is an introductory guide to build hybrid mobile apps using Ionic and Firebase. This book is sample driven. In this book, we are going to build a Hacker News client app. This app can view top stories in Hacker News, view comments of a story, add stories to favorites, etc. This book covers various topics in mobile apps development:

  • Local development environment setup
  • Ionic quickstart
  • Work with Firebase
  • State transition
  • Common UI components: lists, cards, modals, popups
  • Forms & inputs
  • User authentication
  • Publish apps

Source code of the sample app is available on GitHub. View screen-shots of the sample at here.

Buy this book

NodeJS API proxy with CORS support

Our application's backend is Java-based and exports REST API, frontend is AngularJS-based. During frontend development, use Grunt connect to start the development server for CoffeeScript/LESS and static files. To enable AngularJS to access the API which running on different port, we need a proxy with CORS support. So I created a simple proxy server using connect and node-http-proxy.

The proxy code is very simple. In the code below, API server is running on port 8080 and proxy server is running on port 8000. The proxy server sets Access-Control-* headers to enable CORS support. It also provides basic authentication header.

var connect = require('connect'),  
  httpProxy = require('http-proxy');

var app = connect();

var proxy = httpProxy.createProxyServer({  
  target: 'http://127.0.0.1:8080'
});

proxy.on('proxyReq', function(proxyReq, req, res, options) {  
  proxyReq.setHeader('Authorization', 'Basic YWRtaW46cGFzc3dvcmQ=');
});

proxy.on('error', function(e) {  
  console.log(e);
});

app.use(function(req, res, next) {  
  if (req.headers['origin']) {
    res.setHeader('Access-Control-Allow-Origin', req.headers['origin']);
    res.setHeader('Access-Control-Allow-Methods', 'POST, PUT, GET, OPTIONS, DELETE');
    res.setHeader('Access-Control-Max-Age', '3600');
    res.setHeader('Access-Control-Allow-Headers', 'X-Requested-With, Authorization, Content-Type');
  }
  if (req.method !== 'OPTIONS') {
    next();
  }
  else {
    res.end();
  }
});

app.use(function(req, res) {  
  proxy.web(req, res);
});

app.listen(8000);  
console.log('Proxy server started.')  

AngularJS needs to have cross-domain requests enabled.

app.config(function($httpProvider) {  
  $httpProvider.defaults.useXDomain = true
});

Then you should be able to access the API.

Tips for using ProGuard with Spring framework

ProGuard is a is a free Java class file shrinker, optimizer, obfuscator, and preverifier. You may want to use ProGuard to obfuscate your Java binary code first before you release it to customers, especially for Android apps, on-premise enterprise apps or libraries. The whole obfuscation process is very painful and you need to run a lot of tests to make sure your code still works properly after obfuscation.

Here are some tips to use ProGuard, especially when Spring framework is used.

Use the Maven plugin

If you use Maven to manage your project, then you should use the Maven plugin for ProGuard. It's easy to set up and use.

<plugin>  
    <groupId>com.github.wvengen</groupId>
    <artifactId>proguard-maven-plugin</artifactId>
    <version>2.0.10</version>
    <executions>
        <execution>
            <id>proguard</id>
            <phase>package</phase>
            <goals>
                <goal>proguard</goal>
            </goals>
        </execution>
    </executions>
    <configuration>
        <obfuscate>true</obfuscate>
        <injar>${shaded-jar.name}</injar>
        <outjar>${shaded-jar.name}</outjar>
        <libs>
            <lib>${java.bootstrap.classes}</lib>
            <lib>${java.cryptographic.extension.classes}</lib>
            <lib>${java.secure.socket.extension.classes}</lib>
        </libs>
        <injarNotExistsSkip>true</injarNotExistsSkip>
        <options>
        </options>
    </configuration>
    <dependencies>
        <dependency>
            <groupId>net.sf.proguard</groupId>
            <artifactId>proguard-base</artifactId>
            <version>5.2.1</version>
            <scope>runtime</scope>
        </dependency>
    </dependencies>
</plugin>  

In <options> of <configuration>, there should be a list of <option> to configure ProGuard.

Multi-modules project

If your Maven projects have multiple modules, then you should use Maven shade plugin to create a shaded jar for all your modules, then run ProGuard against this single jar. This can make sure ProGuard has the correct mappings for all your application's classes.

<plugin>  
    <groupId>org.apache.maven.plugins</groupId>
    <artifactId>maven-shade-plugin</artifactId>
    <version>2.4</version>
    <configuration>
        <outputFile>${shaded-jar.name}</outputFile>
        <artifactSet>
            <includes>
                <include>com.myapp:*</include>
            </includes>
        </artifactSet>
        <transformers>
            <transformer implementation="org.apache.maven.plugins.shade.resource.AppendingTransformer">
                <resource>META-INF/spring.factories</resource>
            </transformer>
            <transformer implementation="org.apache.maven.plugins.shade.resource.AppendingTransformer">
                <resource>META-INF/spring.handlers</resource>
            </transformer>
            <transformer implementation="org.apache.maven.plugins.shade.resource.AppendingTransformer">
                <resource>META-INF/spring.schemas</resource>
            </transformer>
            <transformer implementation="org.apache.maven.plugins.shade.resource.AppendingTransformer">
                <resource>META-INF/spring.provides</resource>
            </transformer>
        </transformers>
    </configuration>
    <executions>
        <execution>
            <phase>package</phase>
            <goals>
                <goal>shade</goal>
            </goals>
        </execution>
    </executions>
</plugin>  

If you use Spring, make sure transformers are added to process Spring's various files.

Disable optimization class/marking/final

By default ProGuard marks classes as final when possible even when classes are not declared as final. This causes problems for Spring as Spring doesn't allow classes with @Configuration annotation to be final.Use following <option> to disable it.

<option>-optimizations !class/marking/final</option>  

Adapt Spring resources

If you use configuration files like spring.factories to customise Spring, make sure these configuration files are adapted by ProGuard, otherwise the class names in those files will be wrong. META-INF/spring.* in following code specifies Spring configuration files.

<option>-adaptresourcefilecontents **.properties,META-INF/MANIFEST.MF,META-INF/spring.*</option>  

Keep annotations

Spring uses annotations extensively, so annotations should be kept in the runtime to make sure Spring still works properly. *Annotation* in code below is used to keep annotations.

<option>-keepattributes Exceptions,InnerClasses,Signature,Deprecated,SourceFile,LineNumberTable,*Annotation*,EnclosingMethod</option>  

Keep application launch class

If you use Spring Boot, the Application class should be kept to launch the app. The option in code below keeps any class with main method.

<option>-keepclasseswithmembers public class * { public static void main(java.lang.String[]);}</option>  

Keep your REST resources classes

If your app exposes a REST API, make sure those resources classes are kept. Most likely you rely on Jackson or other libraries to convert your resources objects to JSON or XML. These libraries use reflection to find out the properties in your resources classes, so these resources classes should be kept to make sure the JSON/XML representations are correct.

For example, given a resource class User,

public class User {  
    private String firstName;
    private String lastName;

    public String getFirstName() {
        return this.firstName;
    }

    public String getLastName() {
        return this.lastName;
    }
}

After ProGuard processed this class file, the methods getFirstName and getLastName may be renamed to something like a or b. Then Jackson cannot use reflection to find JavaBean properties in this class file. The output will be just an empty JSON object.

<option>-keep public class com.myapp.**.model.** { *; }</option>  

Process bean classes

You can also following examples in ProGuard website to process bean classes by keeping setter and getter methods.

<option>  
-keep class com.myapp.**.model.** {
    void set*(***);
    boolean is*();
    *** get*();
}
</option>  

Add name to Spring beans

If Spring annotations @Service, @Component and @Configuration are used to declare beans, make sure a name is assigned to each bean, e.g. @Component("userHelper") or @Service("userService"). This is because when no name is assigned, Spring uses the class's name as the bean name, but ProGuard will change class names to something like a, b, or c. This will have name conflicts across different packages. For example, package com.myapp.a.a has a class a, package com.myapp.a.b also has a class a, these two class use the same bean name a, but the type is different. So beans should be explicitly named to avoid name conflicts.

Keep members with Spring annotations

If you use Spring annotations like @Value to inject values into your classes like below:

@Value("${myval}")
private String myVal;  

ProGuard is smart enough to infer that the value of myVal is null as this variable has not been assigned any value, so it replaces all occurrences of myVal with null in the binary code, then a lot of NullPointerExceptions will be thrown at runtime. To avoid this, use following options:

<option>-keepclassmembers class * {  
    @org.springframework.beans.factory.annotation.Autowired *;
    @org.springframework.beans.factory.annotation.Value *;
}
</option>  

AngularJS - Features Toggle with Grunt Build

Background

Spring Boot back-end with AngularJS front-end.

Scenario

Our product has two versions: lite version and standard version. Some features are only available in standard version. So some UI components need to be hidden in lite version. This is controlled by build process. By passing different flags to the build process, different versions can be built. Front-end code uses the same flag to show/hide different components.

Solution

Install grunt-ng-constant and load task grunt-ng-constant in Grunt config.

npm install grunt-ng-constant --save-dev  

Then add ngconstant config in Gruntfile. In the config below, I defined two environments, development and production. All environment-related configurations are put into ENV object. distType is the type I want to specify different release versions. In development build, this value is set to standard. In production build, this value is set to grunt.option('distType'), so it's controlled by command line arguments when Grunt is invoked.

ngconstant: {  
  options: {
    space: '  ',
    wrap: 'define(["angular"], function(angular){ \n return {{ "{%= __ngModule " }}%} \n\n });',
    name: 'config'
  },
  development: {
    options: {
      dest: '<%= appConfig.build %>/scripts/config.js'
    },
    constants: {
      ENV: {
        name: 'development',
        apiEndpoint: 'http://localhost:8080/',
        distType: 'standard'
      }
    }
  },
  production: {
    options: {
      dest: '<%= appConfig.build %>/scripts/config.js'
    },
    constants: {
      ENV: {
        name: 'production',
        apiEndpoint: '/',
        distType: grunt.option('distType')
      }
    }
  }
}

Add ngconstant:production to the list of production build tasks. To build a lite version, use grunt build --distType=lite. To build a standard version, use grunt build --distType=standard.

grunt-ng-constant generates the config.js file in specified directory. Include this file using <script> or load it using RequireJS.

define(['angular', 'config'], (angular, config) ->  
  angular.module('myApp', (ENV) ->
    // Use ENV to check version
  )
)

Build Apache Camel Custom Component

If you create a custom Apache Camel component, you can build it using Maven to generate necessary metadata, then this component can be auto-discovered by Camel.

Create a custom component following the guide. Add file META-INF/services/org/apache/camel/component/FOO to src/main/resources folder with content like below:

class=com.example.CustomComponent  

Then add following code to Maven's pom.xml. Maven plugin camel-package-maven-plugin is used to generate component.properties file.

<build>  
    <plugins>
        <plugin>
            <groupId>org.apache.camel</groupId>
            <artifactId>camel-package-maven-plugin</artifactId>
            <version>${camel.version}</version>
            <executions>
                <execution>
                    <goals>
                        <goal>prepare-components</goal>
                    </goals>
                    <phase>generate-resources</phase>
                </execution>
            </executions>
        </plugin>
    </plugins>
</build>  

Then add Maven dependency of org.apache.camel:apt. This apt module processes Camel annotations and generate component JSON schema and HTML documentation. See Camel 2.15 release note.

<dependency>  
    <groupId>org.apache.camel</groupId>
    <artifactId>apt</artifactId>
    <version>${camel.version}</version>
    <scope>provided</scope>
</dependency>  

After this, you should be able to list your component and its JSON schema from JMX.

FileNotFoundException when using Jersey with Spring Boot

When using Jersey with Spring Boot, you may see an FileNotFoundException error like below when Spring Boot starts:

org.springframework.beans.BeanInstantiationException: Failed to instantiate [myapp.JerseyConfig$$EnhancerBySpringCGLIB$$380920c5]:  
Constructor threw exception; nested exception is org.glassfish.jersey.server.internal.scanning.ResourceFinderException:  
java.io.FileNotFoundException: /Users/myapp.jar!/lib/myapp-ws.jar (No such file or directory)  

This error only happens using java -jar to run executable Spring Boot jar file. This is because Spring Boot packages library jars in the lib directory of the executable stand-alone jar. Jersey's class loader cannot scan those jars embedded in the executable jar. In the above error message, Jersey was trying to find myapp-ws.jar on the local file system, but cannot find it because it's embedded in the lib directory of myapp.jar.

To solve this issue, library jars which contain Jersey related classes need to be unpacked first before Spring Boot runs. This is supported by Spring Boot' Maven plugin using requiresUnpack, see doc.

Below is an example of the Maven pom.xml file.

<build>  
    <plugins>
        <plugin>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-maven-plugin</artifactId>
            <configuration>
                <requiresUnpack>
                    <dependency>
                        <groupId>com.myapp</groupId>
                        <artifactId>rest-api</artifactId>
                    </dependency>
                </requiresUnpack>
            </configuration>
        </plugin>
    </plugins>
</build>  

AngularJS - Simple Collapse Directive

Collapse is a common control used in web pages. Users can click to expand or collapse it. Bootstrap has a simple declarative way to create collapse. But Bootstrap's collapse doesn't work if the markup is generated dynamically using AngularJS, because it relies on element id to match target element. For example, following code doesn't work because id is generated dynamically using {{dynamic}}Collapse.

<a class="btn btn-primary" data-toggle="collapse" href="#{{dynamic}}Collapse" aria-expanded="false" aria-controls="{{dynamic}}Collapse">  
Link with href  
</a>  
<div class="collapse" id="{{dynamic}}Collapse">

</div>  

angular-ui collapse directive seems to be a good solution, but it requires new variable in the scope object. So I created a simple directive with jQuery to solve this issue.

angular.module('DemoApp', [])  
.controller('DemoCtrl', ['$scope', function($scope) {
  $scope.colors = ['Red', 'Green', 'Blue'];
}])
.directive('collapseToggler', function(){
  return {
    restrict: 'A'
    link: function(scope, elem, attrs) {
      elem.on('click', function() {
        $(this).siblings('.collapse').toggleClass('in');
      });
    }
  };
})

collapseToggler directive is applied to the toggler. When clicked, it finds the siblings with CSS class collapse and toggle CSS class in which controls display of the target element.

The limitation of this solution is that it requires the target element to be as the sibling of the toggle element. But most of the times this is the desired DOM structure.

Below is an example of how to use it.

<body ng-controller="DemoCtrl">  
  <div ng-repeat="color in colors">
    <div collapse-toggler class="toggler">What's the color?</div>
    <div class="collapse">
    {{color}}
    </div>
  </div>
</body>

See live example:

See the Pen Simple Collapse Directive by Fu Cheng (@alexcheng) on CodePen.

Use Synapse to workaround AWS ELB static IP limitations

Problem

The system receives data from a third-party service using TCP sockets. The service requires a static IP address to send the data. Several app nodes are created to receive and process data. AWS ELB is used as the load balancer for app nodes. But currently AWS ELB only has host name, but no static IP address. AWS has elastic IP addresses which are static, but cannot be associated with ELB.

Solution #1 - HAProxy (Not working)

The first solution I had was to use a HAProxy server as the proxy to AWS ELB. Install HAProxy on one EC2 instance and assign an elastic IP address to it. HAProxy receives data and forward to ELB.

Issue with this solution is that HAProxy only resolves DNS names during start. So once HAProxy starts and IP address of ELB changes, there is no way to detect that and HAProxy keeps sending to old IP address.

Solution #2 - Synapse

Synapse is a service discovery system from Airbnb. It builds upon HAProxy. Synapse provides certain watchers which watch changes. Once changes are detected, Synapse generates a new HAProxy configuration and reloads HAProxy. Application talks to HAProxy instead of the actual proxied service.

Back to the problem, I used Synapse to replace ELB. Synapse has a watcher ec2tag which can watch tags of EC2 instances. To add/remove instances from Synapse, just add/remove certain tags. For example, Synapse watches tag name/value env=test of EC2 instances. Once a new instance with tag env=test is launched, Synapse detects this change and update HAProxy config file to include the new instance. The new instance now is able to receive data. Load balancing is provided by HAProxy.

Install Synapse

It's recommended to install Synapse directly from GitHub master branch. Release 0.11.1 has some issues. For example, if you're using bundler, add following to your Gemfile:

gem 'synapse', :git => 'git://github.com/airbnb/synapse.git'  

If you're using Chef, use gem_specific_install cookbook.

gem_specific_install "synapse" do  
  repository "https://github.com/airbnb/synapse.git"
  revision "master"
  action :install
end  

Configure

Synapse configuration is done by a YAML file synapse.conf.yaml. In this file, you define services and HAProxy configuration.

---
  services:
    myservice:
      default_servers:
        -
          name: "elb"
          host: "<elb-host>"
          port: 7000
      discovery:
        method: "ec2tag"
        tag_name: "env"
        tag_value: "test"
        aws_access_key_id: "<aws-key>"
        aws_secret_access_key: "<aws-secret>"
        aws_region: "<aws-region>"
      haproxy:
        port: 3200
        server_port_override: "7000"
        server_options: "check inter 2000 rise 3 fall 2"
        frontend:
          - "mode tcp"
        backend:
          - "mode tcp"
  haproxy:
    bind_address: "0.0.0.0"
    reload_command: "service haproxy reload"
    config_file_path: "/etc/haproxy/haproxy.cfg"
    do_writes: true
    do_reloads: true
    global:
      - "log 127.0.0.1 local0"
      - "log 127.0.0.1 local1 notice"
      - "user haproxy"
      - "group haproxy"
    defaults:
      - "log global"
      - "balance roundrobin"
      - "timeout client 50s"
      - "timeout connect 5s"
      - "timeout server 50s"

In the Synapse config file above, services section defines different services to watch. For myservice, default_servers section contains the fallback servers when no servers are discovered, here I used the ELB server. discovery section contains configuration about different discovery methods. For ec2tag, you need to provide AWS access key & secret, region and tag name/value to watch. haproxy section contains local HAProxy configuration for this service. In the example, HAProxy for myservice listens on port 3200 and forwards traffic to app nodes no port 7000. The second haproxy section contains global HAProxy configurations.

Run

Copy the YAML file to some place, e.g. /etc/synapse.conf.yaml, then start Synapse using synapse -c /etc/synapse.conf.yaml.

Solution #3 Nginx (untested)

Nginx seems to have better support of DNS resolution, so it may work to use Nginx as the proxy.

Spring RestTemplate Basic Authentication

I'm using Spring RestTemplate to consume REST service with basic authentication, so I need a way to set the username and password. After running some searches, it turns out that it's not that easy to set the username and password directly. So I manually created the Authorization header.

import org.apache.commons.codec.binary.Base64;

HttpHeaders headers = new HttpHeaders();  
headers.set("Authorization", "Basic " + new String(Base64.encodeBase64((username + ":" + password).getBytes(Charset.forName("US-ASCII")))));  
HttpEntity<byte[]> entity = new HttpEntity<byte[]>(headers);  
ResponseEntity<byte[]> response = restTemplate.exchange(uri, HttpMethod.valueOf(httpMethod), entity, byte[].class);