On the fly javascript compression/optimization on nginx using Google Closure Compiler
06 Μαρτίου 2012Google closure compiler allows for smaller javascript files that download and run faster. Serving those optimized javascript files instead of the originals and also turning on gzip compression to serve them can save more than 50% on the original size. Other modern web tools today besides Google closure compiler for optimizing/compressing javascript are Yahoo’s YUI Compressor, Dojo Shrinksafe and UglifyJs.
Indeed the minification process (called compilation on google’s closure compiler site) makes your javascript code quite unreadable. A practical approach is to keep 2 versions of each .js file: the uncompressed version (original source – to work) and the minimized/compressed version ( compiled version – to serve).
Using the nginx web server we can automate the compilation of any javascript source files by using Google Closure Compiler without the need to manually run any external tools. Nginx web server will take care of the compilation, keep the compiled version cached and serve this compiled javascript file on future requests.
In our approach, we want to make use of the embedded Perl module that nginx offers. Several major Linux distributions today offer nginx packages configured with the embedded Perl module.
You can already find an example module to minify javascript files on nginx wiki (EmbeddedPerlMinifyJS) that is using the JavaScript::Minifier Perl CPAN module (inspired by Douglas Crockford’s JSMin). This solution is not optimal (compression/savings achieved) and the CPAN module has not been updated for years (JSMin is about 10 years old). Today you can find actively developed javascript compilers/optimizers that are more sophisticated and evolve. Moreover, in our tests with JavaScript::Minifier some UTF-8 .js files were not minified correctly. Google closure compiler is a better choice today since it is well known, tested, up to date and quite advanced as a javascript compiler/optimizer.
The good news? Google Closure compiler offers a RESTful API…
Even better news? You can find an already working Perl interface for this service on CPAN!
Let’s get started
First, make sure that nginx has been configured and compiled with the Perl module (configured «–with-http_perl_module»). You can run the command «nginx -V» to check. Here is a sample output of the command on a Centos 6 Linux:
# nginx -V
nginx version: nginx/0.8.54
built by gcc 4.4.4 20100726 (Red Hat 4.4.4-13) (GCC)
TLS SNI support enabled
configure arguments: --user=nginx --group=nginx --prefix=/usr/share/nginx
--sbin-path=/usr/sbin/nginx --conf-path=/etc/nginx/nginx.conf
--error-log-path=/var/log/nginx/error.log --http-log-path=/var/log/nginx/access.log
--http-client-body-temp-path=/var/lib/nginx/tmp/client_body
--http-proxy-temp-path=/var/lib/nginx/tmp/proxy
--http-fastcgi-temp-path=/var/lib/nginx/tmp/fastcgi
--http-uwsgi-temp-path=/var/lib/nginx/tmp/uwsgi
--http-scgi-temp-path=/var/lib/nginx/tmp/scgi
--pid-path=/var/run/nginx.pid --lock-path=/var/lock/subsys/nginx
--with-http_ssl_module --with-http_realip_module --with-http_addition_module
--with-http_xslt_module --with-http_image_filter_module --with-http_geoip_module
--with-http_sub_module --with-http_dav_module --with-http_flv_module
--with-http_gzip_static_module --with-http_random_index_module
--with-http_secure_link_module --with-http_degradation_module --with-http_stub_status_module
--with-http_perl_module --with-mail --with-file-aio --with-mail_ssl_module --with-ipv6
--with-cc-opt='-O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector
--param=ssp-buffer-size=4 -m64 -mtune=generic' --with-cc-opt='-O2 -g -pipe -Wall
-Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4
-m64 -mtune=generic'
As you can see above our nginx there is already configured «–with-http_perl_module». Also note on the first line the –prefix (it is /usr/share/nginx), we will use it in a while.
Now, you should install WebService::Google::Closure from CPAN on your system or do a manual module installation.
A quick note for those not familiar with CPAN (assuming a Linux install): Enter «perl -MCPAN -e shell» as root (or the command «cpan»). If this is your first time running cpan there might be some setup questions. When you get the cpan shell, enter: «install WebService::Google::Closure». Say yes to install any required depedencies on your system. After installing the module type «exit» to return to your shell.
The Perl handler
As a next step, let’s write our handler module. We will create the new file (Perl code) in a subdirectory of the —prefix directory mentioned above (actually we will create a new directory «perl_handlers» there just to keep things tidy).
File JSClosureCompiler.pm (save into /usr/share/nginx/perl_handlers):
package JSClosureCompiler;
use nginx;
use WebService::Google::Closure;
#to make sure we write UTF-8 files
use open OUT => ':utf8';
sub handler {
my $r=shift;
my $cache_dir="/tmp"; # Cache directory where optimized files will be kept
my $cache_file=$r->uri;
$cache_file=~s!/!_!g;
$cache_file=join("/", $cache_dir, $cache_file);
my $uri=$r->uri;
my $filename=$r->filename;
return DECLINED unless -f $filename;
if (! -f $cache_file) {
my $minjs = WebService::Google::Closure->new(file => $filename)->compile;
open(OUTFILE, '>' . $cache_file ) or die "Error writting file: $!";
print OUTFILE $minjs->code;
close(OUTFILE);
}
$r->send_http_header("application/javascript");
$r->sendfile($cache_file);
return OK;
}
1;
__END__
As you can see the optimized files are cached in $cache_dir («/tmp» here) to avoid optimizing the same files again in every web request (of course this should be avoided – better serve your uncompressed .js files in that case!). An underscore character is prepended on cached files filenames.
If you need to regenerate an optimized javascript file (after updating your javascript source code for example), just delete the cached file from /tmp (it will be regenerated fresh on the next request of this file by nginx).
The nginx side
To use our handler on nginx, you should update your nginx.conf:
http {
perl_modules perl_handlers; #our newly created directory
perl_require WebService/Google/Closure.pm;
perl_require JSClosureCompiler.pm;
root /var/www;
server {
location / {
....
}
location ~ \.js$ {
perl JSClosureCompiler::handler;
}
}
}
Magic happens of course at the location ~ .js$ {…} block, where we pass all our .js files to our custom Perl handler. Reload nginx afterwards to reread your configuration.
You can now load a .js file on your browser and enjoy freshly optimized javascript being served. Of course you should test any applications you may already serve with the optimized javascript.
How to control the aggressiveness of the javascript compiler
You can control how aggressive the javascript compiler will be. The line:
my $minjs = WebService::Google::Closure->new(file => $filename)->compile;uses the default ‘simple optimizations’ as defined by Google Closure Optimizer.
For whitespace removal only, you can use:
my $minjs = WebService::Google::Closure->new( file => $filename, compilation_level => 1, )->compile;and for «advanced optimizations» just change the compilation_level above to 3.
I try to syntax check my module by using «perl -c» and I get an error about an «undefined symbol».
In syntax checking of Perl scripts all «use» statements are still executed. Do not «use nginx;» outside nginx. Everything will be ok when running inside nginx. If this is the only error you get, go ahead and try it inside nginx.