Lisplets - a Lisp-friendly interface to Java Servlets

Copyright (c) Rich Hickey. All rights reserved.

The use and distribution terms for this software are covered by the Common Public License 1.0, which can be found in the file CPL.TXT at the root of this distribution. By using this software in any fashion, you are agreeing to be bound by the terms of this license. You must not remove this notice, or any other, from this software.

Contents

Introduction

Lisplets are Java Servlets that forward their requests, and gather their response headers, using s-expressions over sockets. They enable easy integration of Common Lisp or Scheme into a Java-based web environment.

The library consists of a single Java class which implements HTTPServlet. It packages the request header, parameters, cookies, session, application, and user info into an s-expression and sends it over a TCP socket that can be read from Lisp using read. The Lisp process returns status, headers, session and application state as an s-expression, followed by the document content. This servlet should work with any servlet container, such as Tomcat, Resin, Jetty, JRun and many others.

The advantages for Lisp developers are plentiful. The various servlet container implementations span the full range of cost (including free), embed-ability, scalability, platform support etc. Several provide a full HTTP server implementation and all can integrate with the industry standard web servers. Thus the Lisp developer is spared the re-inventing of an HTTP server, SSL support etc. In addition, these platforms support the sending of only specific requests to the servlet, allowing for static content to be handled directly by the web server, or a mix of Lisp-generated content and JSP content. The servlet API goes beyond HTTP services by providing for session management, authentication, access control, user roles, session and application state, declarative configuration and logging, all of which are reflected in the Lisplet API. In addition to the convenience, this means that Lisp application servers can interoperate with Java Server Pages, Java Servlets and other Java-based web technologies in a single web application.

Download

The Lisplets project is hosted on SourceForge SourceForge.net Logo

Setup and Configuration

The Lisplet servlet is supplied as a single classfile: com/richhickey/lisplets/Lisplet.class. While a full description of web app deployment is beyond the scope of this document, the basic idea is to set up either a directory or .WAR file that contains a WEB-INF directory. In the WEB-INF directory should be a web.xml file that contains the application configuration, (a sample web.xml file is included), and a WEB-INF/classes/ directory, into which the entire path com/richhickey/lisplets/Lisplet.class should be placed.

All requests to URIs matching the pattern specified in the servlet-mapping section of web.xml will be routed to the Lisplet and thus to the host and port specified by the lisp-host and lisp-port init-params. If you are unfamiliar with servlets I recommend you install Tomcat and read their documentation.

Quick Start

Once you have a directory structure or .WAR file you need to follow the instructions of your servlet container implementation for deploying the web app. This may require copying the directory or .WAR file and/or restarting the servlet container. Tomcat 5, with which I developed Lisplets, has a nice HTTP front end for their management utility that let you deploy from the browser without restarting Tomcat.

At this point, all you need is a Lisp/Scheme process to service the requests. The request is sent as a single s-expression. The reply must be a single s-expression containing headers etc, followed by the body content, in any form, which will be transferred byte-for-byte to the client. Close the socket to conclude the response. Here is a sample server that simply pretty-prints the request it received as the body of the response. It was written for Xanalys LispWorks, and the socket and process portion will need to be modified for your specific Lisp/socket implementation:

(require "comm")

(defvar *lisplet-port* 13579) ; a free TCP port number

(defun make-lisplet-stream (handle)
  (let ((stream (make-instance 'comm:socket-stream
                               :socket handle
                               :direction :io
                               :element-type 'base-char)))
    (mp:process-run-function (format nil "lisplet ~D"
                                     handle)
                             '()
                             'handle-lisplet-on-stream stream)))

(defun handle-lisplet-on-stream (stream)
  (unwind-protect
      (let ((req (read stream)))
        ;set something in the return header
        (format stream "~S"
                `((:status 200)
                  (:headers
                   (:content-type "text/plain"))
                  (:cookies
                   ((:name "lisp-cookie")
                    (:value ,(second (assoc :uri req)))))
                  (:session
                   (:max-inactive-interval 60)
                   (:attrs
                    (:fred "ethel")
                    (:lucy "ricky")))
                  (:context
                   (:attrs
                    (:com.richhickey.foo "bar")))
                  (:log "Hello from Lisp")))
        ;echo the request as the body
        (pprint req stream)
        (force-output stream)
        (close stream))))

(comm:start-up-server :function 'make-lisplet-stream
                      :service *lisplet-port*)
	

When run under Tomcat (with an authenticated user) this produces a page like this (i.e., this is what is being sent to Lisp):

((:METHOD "GET")
 (:URI "/lisplets/test.do")
 (:PROTOCOL "HTTP/1.1")
 (:IS-SECURE NIL)
 (:HEADERS
  (:HOST "127.0.0.1:8080")
  (:USER-AGENT
   "Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.6) Gecko/20040113")
  (:ACCEPT
   "application/x-shockwave-flash,text/xml,application/xml,application/xhtml+xml,text/html;q=0.9,text/plain;q=0.8,image/png,image/jpeg,image/gif;q=0.2,*/*;q=0.1")
  (:ACCEPT-LANGUAGE "en-us,en;q=0.5")
  (:ACCEPT-ENCODING "gzip,deflate")
  (:ACCEPT-CHARSET "ISO-8859-1,utf-8;q=0.7,*;q=0.7")
  (:KEEP-ALIVE "300")
  (:CONNECTION "keep-alive")
  (:AUTHORIZATION "Basic YWRtaW46aW50YWN0"))
 (:COOKIES
  (:LISP-COOKIE "/lisplets/test.do")
  (:JSESSIONID "8D1036E516561CC1A8A146916591BAD4"))
 (:PARAMS (:FRED "ethel"))
 (:CONTEXT
  (:NAME "Lisplets Test")
  (:PARAMS (:A-CONTEXT-PARAM "lisplet-context-param"))
  (:ATTRS
   (:JAVAX.SERVLET.CONTEXT.TEMPDIR
    "C:\\Program Files\\Apache Software Foundation\\Tomcat 5.0\\work\\Catalina\\localhost\\lisplets")
   (:ORG.APACHE.CATALINA.RESOURCES
    "org.apache.naming.resources.ProxyDirContext@17e4dee")
   (:ORG.APACHE.CATALINA.WELCOME_FILES "[Ljava.lang.String;@12e7c6a")))
 (:SERVLET
  (:NAME "lisplet")
  (:PARAMS
   (:LISP-ROLES "role1,tomcat,admin")
   (:SAMPLE-USER-PARAM "a param")
   (:LISP-CREATE-SESSION "true")
   (:LISP-TAG-PREFIX ":")
   (:LISP-PORT "13579")
   (:LISP-HOST "localhost")))
 (:SESSION
  (:ID "566E6A3F88B70901B68B2AB0420D84E6")
  (:IS-NEW T)
  (:CREATION-TIME 1096421920743)
  (:LAST-ACCESSED-TIME 1096421920743)
  (:MAX-INACTIVE-INTERVAL 1800))
 (:USER
  (:NAME "admin")
  (:AUTH-TYPE "BASIC")
  (:ROLES (:ROLE1 T) (:TOMCAT T) (:ADMIN T))))
	

Hopefully that looks simple and natural to a Lisp developer.

API Reference

web.xml Parameters

Several init-params are used to configure the operation of the Lisplet servlet. lisp-host and lisp-port are required, all others are optional.

Request Format

All information about the request is sent in a single list. All members of that list are themselves lists, each beginning with a tag. The top-level entries are:

The entries are not guaranteed to come in any particular order, nor are they all guaranteed to be present. Basically, if there is nothing to send the entry is omitted, e.g. if the user is not logged in there is no user entry, if there is no session there will be no session entry, no cookies, no cookie entry etc. Unless otherwise specified, strings have been URL-decoded. Strings are sent as " - quoted literals. Only " and \ are escaped, by prefixing with \

method, uri, protocol and is-secure

The result of getMethod(), getRequestURI(), getProtocol() and isSecure() respectively. Only the GET and POST methods are currently supported.

headers

All of the http headers are included except any cookies, which are in a separate section. Note that a header may have multiple values.

cookies

All of the cookies are included. Note that this is asymmetric vs the response cookie format, which allows you to set a comment, domain, path etc. While these other cookie attributes can be sent to a client, they are not returned.

params

All of the query parameters, either from the query string or posted form data, are included. Note that a parameter may have multiple values.

context

The nested entries are:

servlet

The nested entries are:

session

The nested entries are:

user

The nested entries are:

Response Format

The response consists of two parts, an s-expression containing header, cookie and other directives to the servlet, followed by the body. Even if you have no headers etc to send (there are several defaults, e.g. Status 200), you must send at least the empty list (). The response header s-expression is read by a very simplistic reader on the Java side. The reader uses only (, ), and whitespace as delimiters. It can read " - quoted string literals, and will interpret only \" and \\ escapes in string literals. It can read nested lists. Anything other than a list or string literal will be read in as a whitespace/paren delimited string. If a value needs to be treated as a string, it will be compared to the lisp-null string, and, if it matches, null will be used, otherwise, with no translation other than URL-encoding, if necessary. If the value needs to be treated as a number, Integer.parseInt() will be used. If it needs to be treated as a boolean, it will be compared to the lisp-true string, and the result of that comparison will be used.

All information about the response is sent in a single list. All members of that list are themselves lists, each beginning with a tag. The top-level entries are:

All top-level entries are optional, and may appear in any order, but not more than once.

status, error, redirect

These result in calls to setStatus(), sendError(), sendRedirect() respectively. Only one of these should be sent. error and redirect will cause the rest of the response to be ignored. If none of these are sent, the status defaults to 200 (Ok).

headers

Any standard http headers except cookies, which are more conveniently sent with a cookies entry.

cookies

Each cookie is a list, with the following entries: See the documentation for the Cookie class for descriptions of these attributes.

context

The nested entries are:

session

The nested entries are: Note that if max-inactive-interval or any attrs are specified, a session will be created if one does not exist, regardless of the setting of lisp-create-session.

log

Each string will be sent to log().

Summary

This first version of Lisplets supports only standard gets with query-strings and form-style posts. Future plans include support for custom posts, and socket keep-alive between the Java servlet and Lisp.

I hope you find Lisplets useful. I welcome comments and code contributions.

Rich Hickey, September 2004