The use and distribution terms for this software are covered by the Common Public License 1.0, which can be found in the file CPL.TXT at the root of this distribution. By using this software in any fashion, you are agreeing to be bound by the terms of this license. You must not remove this notice, or any other, from this software.
Lisplets are Java Servlets that forward their requests, and gather their response headers, using s-expressions over sockets. They enable easy integration of Common Lisp or Scheme into a Java-based web environment.
The library consists of a single Java class which implements
HTTPServlet.
It packages the request header, parameters, cookies, session, application, and
user info into an s-expression and sends it over a TCP socket that can be read
from Lisp using read. The Lisp process returns status, headers,
session and application state as an s-expression, followed by the document
content. This servlet should work with any servlet container, such as
Tomcat,
Resin, Jetty,
JRun and many others.
The advantages for Lisp developers are plentiful. The various servlet container implementations span the full range of cost (including free), embed-ability, scalability, platform support etc. Several provide a full HTTP server implementation and all can integrate with the industry standard web servers. Thus the Lisp developer is spared the re-inventing of an HTTP server, SSL support etc. In addition, these platforms support the sending of only specific requests to the servlet, allowing for static content to be handled directly by the web server, or a mix of Lisp-generated content and JSP content. The servlet API goes beyond HTTP services by providing for session management, authentication, access control, user roles, session and application state, declarative configuration and logging, all of which are reflected in the Lisplet API. In addition to the convenience, this means that Lisp application servers can interoperate with Java Server Pages, Java Servlets and other Java-based web technologies in a single web application.
The Lisplets project is
hosted on SourceForge
The Lisplet servlet is supplied as a single classfile: com/richhickey/lisplets/Lisplet.class.
While a full description of web app deployment is beyond the scope of this
document, the basic idea is to set up either a directory or .WAR file that
contains a WEB-INF directory. In the WEB-INF directory
should be a web.xml file that contains the application
configuration, (a sample web.xml file is
included), and a WEB-INF/classes/ directory, into which the entire
path com/richhickey/lisplets/Lisplet.class should be placed.
All requests to URIs matching the pattern specified in the servlet-mapping
section of web.xml will be routed to the Lisplet and thus to the host and port
specified by the lisp-host and lisp-port init-params. If you are unfamiliar with
servlets I recommend you install Tomcat
and read their documentation.
Once you have a directory structure or .WAR file you need to follow the instructions of your servlet container implementation for deploying the web app. This may require copying the directory or .WAR file and/or restarting the servlet container. Tomcat 5, with which I developed Lisplets, has a nice HTTP front end for their management utility that let you deploy from the browser without restarting Tomcat.
At this point, all you need is a Lisp/Scheme process to service the requests. The request is sent as a single s-expression. The reply must be a single s-expression containing headers etc, followed by the body content, in any form, which will be transferred byte-for-byte to the client. Close the socket to conclude the response. Here is a sample server that simply pretty-prints the request it received as the body of the response. It was written for Xanalys LispWorks, and the socket and process portion will need to be modified for your specific Lisp/socket implementation:
(require "comm")
(defvar *lisplet-port* 13579) ; a free TCP port number
(defun make-lisplet-stream (handle)
(let ((stream (make-instance 'comm:socket-stream
:socket handle
:direction :io
:element-type 'base-char)))
(mp:process-run-function (format nil "lisplet ~D"
handle)
'()
'handle-lisplet-on-stream stream)))
(defun handle-lisplet-on-stream (stream)
(unwind-protect
(let ((req (read stream)))
;set something in the return header
(format stream "~S"
`((:status 200)
(:headers
(:content-type "text/plain"))
(:cookies
((:name "lisp-cookie")
(:value ,(second (assoc :uri req)))))
(:session
(:max-inactive-interval 60)
(:attrs
(:fred "ethel")
(:lucy "ricky")))
(:context
(:attrs
(:com.richhickey.foo "bar")))
(:log "Hello from Lisp")))
;echo the request as the body
(pprint req stream)
(force-output stream)
(close stream))))
(comm:start-up-server :function 'make-lisplet-stream
:service *lisplet-port*)
When run under Tomcat (with an authenticated user) this produces a page like this (i.e., this is what is being sent to Lisp):
((:METHOD "GET")
(:URI "/lisplets/test.do")
(:PROTOCOL "HTTP/1.1")
(:IS-SECURE NIL)
(:HEADERS
(:HOST "127.0.0.1:8080")
(:USER-AGENT
"Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.6) Gecko/20040113")
(:ACCEPT
"application/x-shockwave-flash,text/xml,application/xml,application/xhtml+xml,text/html;q=0.9,text/plain;q=0.8,image/png,image/jpeg,image/gif;q=0.2,*/*;q=0.1")
(:ACCEPT-LANGUAGE "en-us,en;q=0.5")
(:ACCEPT-ENCODING "gzip,deflate")
(:ACCEPT-CHARSET "ISO-8859-1,utf-8;q=0.7,*;q=0.7")
(:KEEP-ALIVE "300")
(:CONNECTION "keep-alive")
(:AUTHORIZATION "Basic YWRtaW46aW50YWN0"))
(:COOKIES
(:LISP-COOKIE "/lisplets/test.do")
(:JSESSIONID "8D1036E516561CC1A8A146916591BAD4"))
(:PARAMS (:FRED "ethel"))
(:CONTEXT
(:NAME "Lisplets Test")
(:PARAMS (:A-CONTEXT-PARAM "lisplet-context-param"))
(:ATTRS
(:JAVAX.SERVLET.CONTEXT.TEMPDIR
"C:\\Program Files\\Apache Software Foundation\\Tomcat 5.0\\work\\Catalina\\localhost\\lisplets")
(:ORG.APACHE.CATALINA.RESOURCES
"org.apache.naming.resources.ProxyDirContext@17e4dee")
(:ORG.APACHE.CATALINA.WELCOME_FILES "[Ljava.lang.String;@12e7c6a")))
(:SERVLET
(:NAME "lisplet")
(:PARAMS
(:LISP-ROLES "role1,tomcat,admin")
(:SAMPLE-USER-PARAM "a param")
(:LISP-CREATE-SESSION "true")
(:LISP-TAG-PREFIX ":")
(:LISP-PORT "13579")
(:LISP-HOST "localhost")))
(:SESSION
(:ID "566E6A3F88B70901B68B2AB0420D84E6")
(:IS-NEW T)
(:CREATION-TIME 1096421920743)
(:LAST-ACCESSED-TIME 1096421920743)
(:MAX-INACTIVE-INTERVAL 1800))
(:USER
(:NAME "admin")
(:AUTH-TYPE "BASIC")
(:ROLES (:ROLE1 T) (:TOMCAT T) (:ADMIN T))))
Hopefully that looks simple and natural to a Lisp developer.
lisp-host and lisp-port
are required, all others are optional.
A string. The host to which the socket connection will be made. Default: none
An integer. The port to which the socket connection will be made. Default: none
A string. This string will be prepended to all 'tags', which means header, init-param, attribute, cookie and param names. For example, specifying a prefix of : will cause those names to appear as keywords to Common Lisp. The prefix works in both directions, meaning 'tags' sent from Lisp to Java will have this prefix removed before being used as header, cookie etc names. Default: none
A string. This string will be appended to all 'tags', which means header, init-param, attribute, cookie and param names. The suffix works only one way (Java to Lisp), and the only conceived-of use at this time is to specify a prefix and suffix of " in order to have tags sent to Lisp as strings (vs. symbols). Default: none
true or false. If true, the Lisplet will create a session if none exists. Default: false
A comma-separated list of role names. If authentication is configured and the user is logged in, the Lisplet will inquire of the user's membership in the listed roles. Authorization and roles must be configured in the standard manner in the web.xml file. See the documentation for your servlet container for your options for a user and role database. Tomcat lets you get a quick start by using a simple xml file. Default: none
A string. The symbol to use to communicate true to/from Lisp. Default: t
A string. The symbol to use to communicate false to/from Lisp. Default: nil
A string. The symbol to use to communicate null to/from Lisp. Default: nil
method a-string)
uri a-string)
protocol a-string)
is-secure a-boolean)
headers
cookies
params
context
name a-string)
params (name value) ...)
attrs (name value) ...))
servlet
name a-string)
params (name value) ...))
session id a-string)
is-new a-boolean)
creation-time an-int)
last-accessed-time an-int)
max-inactive-interval an-int)
attrs (name value) ...))
user
name a-string)
auth-type a-string)
roles (name a-boolean) ...))
The entries are not guaranteed to come in any particular order, nor are they all guaranteed to be present. Basically, if there is nothing to send the entry is omitted, e.g. if the user is not logged in there is no user entry, if there is no session there will be no session entry, no cookies, no cookie entry etc. Unless otherwise specified, strings have been URL-decoded. Strings are sent as " - quoted literals. Only " and \ are escaped, by prefixing with \
display-name
entry in web.xml.
servlet-name
entry in web.xml.
user
section.
lisp-roles servlet init parameter.
The response consists of two parts, an s-expression containing header, cookie
and other directives to the servlet, followed by the body. Even if you have no
headers etc to send (there are several defaults, e.g. Status 200), you must send
at least the empty list (). The response header s-expression is read by a very
simplistic reader on the Java side. The reader uses only (, ), and whitespace as
delimiters. It can read " - quoted string literals, and will interpret only \"
and \\ escapes in string literals. It can read nested lists. Anything other than
a list or string literal will be read in as a whitespace/paren delimited string.
If a value needs to be treated as a string, it will be compared to the
lisp-null string, and, if it matches, null will be
used, otherwise, with no translation other than URL-encoding, if necessary. If the value needs to be treated as a
number, Integer.parseInt()
will be used. If it needs to be treated as a boolean, it will be compared to the
lisp-true string, and the result of that comparison will be
used.
All information about the response is sent in a single list. All members of that list are themselves lists, each beginning with a tag. The top-level entries are:
status an-int)
error an-int [a-message-string])
redirect a-location-URL)
headers
cookies name a-string) (value a-string)comment a-string) (domain a-string) (max-age
an-int) (path a-string) (secure a-boolean) (version
an-int)]) ...)
context
attrs (name value) ...))
session invalidate)
max-inactive-interval an-int)
attrs (name value) ...))
log a-string ...)
error and redirect will
cause the rest of the response to be ignored. If none of these are sent, the status defaults to 200 (Ok).
lisp-null)
is a possible value for an attribute, in which case the attribute will be removed (i.e. it is not null, it is gone).
lisp-null)
is a possible value for an attribute, in which case the attribute will be removed (i.e. it is not null, it is gone).
This first version of Lisplets supports only standard gets with query-strings and form-style posts. Future plans include support for custom posts, and socket keep-alive between the Java servlet and Lisp.
I hope you find Lisplets useful. I welcome comments and code contributions.
Rich Hickey, September 2004