4.1 What makes Lisp different

The following sums up how bindings to foreign libraries are usually implemented in other languages, then in Common Lisp:

Perl, Python, Java, other one-implementation languages

Bindings are implemented as shared objects written in C. In some cases, the C code is generated by a tool, such as SWIG, but the result is the same: a new C library that manually translates between the language implementation’s objects, such as PyObject in Python, and whatever C object is called for, often using C functions provided by the implementation. It also translates between the calling conventions of the language and C.

Common Lisp

Bindings are written in Lisp. They can be created at-will by Lisp programs. Lisp programmers can write new bindings and add them to the image, using a listener such as SLIME, as easily as with regular Lisp definitions. The only foreign library to load is the one being wrapped—the one with the pure C interface; no C or other non-Lisp compilation is required.

We believe the advantages of the Common Lisp approach far outweigh any disadvantages. Incremental development with a listener can be as productive for C binding development as it is with other Lisp development. Keeping it “in the [Lisp] family”, as it were, makes it much easier for you and other Lisp programmers to load and use the bindings. Common Lisp implementations such as CMUCL, freed from having to provide a C interface to their own objects, are thus freed to be implemented in another language (as CMUCL is) while still allowing programmers to call foreign functions.

Perhaps the greatest advantage is that using an FFI doesn’t obligate you to become a professional binding developer. Writers of bindings for other languages usually end up maintaining or failing to maintain complete bindings to the foreign library. Using an FFI, however, means if you only need one or two functions, you can write bindings for only those functions, and be assured that you can just as easily add to the bindings if need be.

The removal of the C compiler, or C interpretation of any kind, creates the main disadvantage: some of C’s “abstractions” are not available, violating information encapsulation. For example, structs that must be passed on the stack, or used as return values, without corresponding functional abstractions to create and manage the structs, must be declared explicitly in Lisp. This is fine for structs whose contents are “public”, but is not so pleasant when a struct is supposed to be “opaque” by convention, even though it is not so defined.¹

Without an abstraction to create the struct, Lisp needs to be able to lay out the struct in memory, so must know its internal details.

In these cases, you can create a minimal C library to provide the missing abstractions, without destroying all the advantages of the Common Lisp approach discussed above. In the case of structs, you can write simple, pure C functions that tell you how many bytes a struct requires or allocate new structs, read and write fields of the struct, or whatever operations are supposed to be public.² The Groveller automates this and other processes.

Another disadvantage appears when you would rather use the foreign language than Lisp. However, someone who prefers C to Lisp is not a likely candidate for developing a Lisp interface to a C library.