LLVM / JIT Support in Mac OS X

Just In Time (JIT) compiler using LLVM backend is an experimental feature in Octave, which will make Octave capable of compiling during run-time. It can significantly speed up loops and even other parts of the program as well. The development of JIT for Octave is beeyond the scope of my project, and at it is a separate GSoC project of some other student. Here is how I added JIT support in the Octave build.

Natively compiling LLVM using MXE and then using it to compile Octave is the standard way to get JIT support. This process has been followed to compile Octave for MinGW. However I saw that there are many discrepancies while following this process in Mac OS X. If you read the previous post in this blog, I had stated about the problems I had faced with dynamic library path names. I saw that there is a similar problem with the llvm-config utility and it turned out to be a self-referential symbolic link.
This is the result of ls -l ./usr/bin/llvm-config
lrwxr-xr-x  1 root  staff  48 Sep 11 02:29 ./usr/bin/llvm-config -> /Users/mac6-user1/mxe-octave/usr/bin/llvm-config

On a Mac OS X system, there is no special need to compile LLVM separately because it is the default compiler present. The llvm-config utility happens to be installed in the path /opt/local/libexec/llvm-3.3/bin/llvm-config. I added this path in the environment variable LLVM_CONFIG and explicitly specified the --enable-jit option in the configure and Octave successfully compiled with JIT. Here are the configure details.

LLVM CPPFLAGS:               -isystem /opt/local/libexec/llvm-3.3/include
LLVM LDFLAGS:                -L/opt/local/libexec/llvm-3.3/lib
LLVM libraries:              -lLLVMInstrumentation -lLLVMArchive -lLLVMLinker -lLLVMIRReader -lLLVMBitReader -lLLVMAsmParser -lLLVMDebugInfo -lLLVMOption -lLLVMipo -lLLVMVectorize -lLLVMBitWriter -lLLVMTableGen -lLLVMSystemZCodeGen -lLLVMSystemZAsmParser -lLLVMSystemZDesc -lLLVMSystemZInfo -lLLVMSystemZAsmPrinter -lLLVMHexagonCodeGen -lLLVMHexagonAsmPrinter -lLLVMHexagonDesc -lLLVMHexagonInfo -lLLVMNVPTXCodeGen -lLLVMNVPTXDesc -lLLVMNVPTXInfo -lLLVMNVPTXAsmPrinter -lLLVMMBlazeDisassembler -lLLVMMBlazeCodeGen -lLLVMMBlazeDesc -lLLVMMBlazeAsmPrinter -lLLVMMBlazeAsmParser -lLLVMMBlazeInfo -lLLVMCppBackendCodeGen -lLLVMCppBackendInfo -lLLVMMSP430CodeGen -lLLVMMSP430Desc -lLLVMMSP430Info -lLLVMMSP430AsmPrinter -lLLVMXCoreDisassembler -lLLVMXCoreCodeGen -lLLVMXCoreDesc -lLLVMXCoreInfo -lLLVMXCoreAsmPrinter -lLLVMMipsDisassembler -lLLVMMipsCodeGen -lLLVMMipsAsmParser -lLLVMMipsDesc -lLLVMMipsInfo -lLLVMMipsAsmPrinter -lLLVMARMDisassembler -lLLVMARMCodeGen -lLLVMARMAsmParser -lLLVMARMDesc  -lLLVMARMInfo -lLLVMARMAsmPrinter -lLLVMAArch64Disassembler -lLLVMAArch64CodeGen -lLLVMAArch64AsmParser -lLLVMAArch64Desc -lLLVMAArch64Info -lLLVMAArch64AsmPrinter -lLLVMAArch64Utils -lLLVMPowerPCCodeGen -lLLVMPowerPCDesc -lLLVMPowerPCAsmPrinter -lLLVMPowerPCAsmParser -lLLVMPowerPCInfo -lLLVMSparcCodeGen -lLLVMSparcDesc -lLLVMSparcInfo -lLLVMX86Disassembler -lLLVMX86AsmParser -lLLVMX86CodeGen -lLLVMSelectionDAG -lLLVMAsmPrinter -lLLVMX86Desc -lLLVMX86Info -lLLVMX86AsmPrinter -lLLVMX86Utils -lLLVMMCDisassembler -lLLVMMCParser -lLLVMInterpreter -lLLVMMCJIT -lLLVMJIT -lLLVMCodeGen -lLLVMObjCARCOpts -lLLVMScalarOpts -lLLVMInstCombine -lLLVMTransformUtils -lLLVMipa -lLLVMAnalysis -lLLVMRuntimeDyld -lLLVMExecutionEngine -lLLVMTarget -lLLVMMC -lLLVMObject -lLLVMCore -lLLVMSupport

JIT compiler for loops:             yes

The final evaluation is close and since I have already a working build of Octave for Mac OS X, I will push all my code to my repo mxe-octave-anirudha now.

Support for sparse matrices

It took me some time to figure out a way to enable functionality for sparse matrices in Octave. Initially, the configure script was unable to find the location where the sparse matrix libraries are present. I had to specify the paths in src/octave.mk like this:

--with-umfpack-includedir='$(HOST_INCDIR)' \
--with-umfpack-libdir='$(HOST_LIBDIR)' \
--with-cxsparse-includedir='$(HOST_INCDIR)' \
--with-cxsparse-libdir='$(HOST_LIBDIR)' \

Manually specifying the paths worked, and Octave was built with the appropriate CPPFLAGS and LDFLAGS. The configure details of the libraries which add the functionality of sparse matrices in Octave are given below:

AMD CPPFLAGS: -I/Users/mac6-user1/mxe/mxe-octave-anirudha/usr/x86_64-apple-darwin12.2.1/include
AMD LDFLAGS: -L/Users/mac6-user1/mxe/mxe-octave-anirudha/usr/x86_64-apple-darwin12.2.1/lib
AMD libraries: -lamd

CAMD CPPFLAGS: -I/Users/mac6-user1/mxe/mxe-octave-anirudha/usr/x86_64-apple-darwin12.2.1/include
CAMD LDFLAGS: -L/Users/mac6-user1/mxe/mxe-octave-anirudha/usr/x86_64-apple-darwin12.2.1/lib
CAMD libraries: -lcamd

CCOLAMD CPPFLAGS: -I/Users/mac6-user1/mxe/mxe-octave-anirudha/usr/x86_64-apple-darwin12.2.1/include
CCOLAMD LDFLAGS: -L/Users/mac6-user1/mxe/mxe-octave-anirudha/usr/x86_64-apple-darwin12.2.1/lib
CCOLAMD libraries: -lccolamd

COLAMD CPPFLAGS: -I/Users/mac6-user1/mxe/mxe-octave-anirudha/usr/x86_64-apple-darwin12.2.1/include
COLAMD LDFLAGS: -L/Users/mac6-user1/mxe/mxe-octave-anirudha/usr/x86_64-apple-darwin12.2.1/lib
COLAMD libraries: -lcolamd

CXSPARSE CPPFLAGS: -I/Users/mac6-user1/mxe/mxe-octave-anirudha/usr/x86_64-apple-darwin12.2.1/include
CXSPARSE LDFLAGS: -L/Users/mac6-user1/mxe/mxe-octave-anirudha/usr/x86_64-apple-darwin12.2.1/lib
CXSPARSE libraries: -lcxsparse

UMFPACK CPPFLAGS: -I/Users/mac6-user1/mxe/mxe-octave-anirudha/usr/x86_64-apple-darwin12.2.1/include
UMFPACK LDFLAGS: -L/Users/mac6-user1/mxe/mxe-octave-anirudha/usr/x86_64-apple-darwin12.2.1/lib
UMFPACK libraries: -lumfpack

CHOLMOD CPPFLAGS: -I/Users/mac6-user1/mxe/mxe-octave-anirudha/usr/x86_64-apple-darwin12.2.1/include
CHOLMOD LDFLAGS: -L/Users/mac6-user1/mxe/mxe-octave-anirudha/usr/x86_64-apple-darwin12.2.1/lib
CHOLMOD libraries: -lcholmod

Even if Octave is built this way, you are likely to see some errors like the one below:

dyld: Library not loaded: libcholmod.so
Referenced from: /Users/mac6-user1/mxe/mxe-octave-anirudha/usr/x86_64-apple-darwin12.2.1/bin/octave
Reason: image not found
Trace/BPT trap: 5

The hint to solve this came from Alexander Hansen. He suggested that /Users/mac6-user1/mxe/mxe-octave-anirudha/usr/x86_64-apple-darwin12.2.1/bin/octave wants to link “libcholmod.so”, which would be /Users/mac6-user1/mxe/mxe-octave-anirudha/usr/x86_64-apple-darwin12.2.1/lib/libcholmod.so . When I did otool -L /Users/mac6-user1/mxe/mxe-octave-anirudha/usr/x86_64-apple-darwin12.2.1/bin/octave, I saw that the sparse matrix libraries like libcholmod.so and friends had just the names in the list of links, which should have been actual absolute paths. “otool” is a command that displays specified parts of object files or libraries.

One solution which seemed valid to me was to change the install names of these dynamic shared libraries. Fortunately there is already a tool in Mac OS X called “install_name_tool” to do this job. The dynamic library libcholmod.so can be made visible to octave by the following command:

install_name_tool -change "libcholmod.so" "/Users/mac6-user1/mxe/mxe-octave-anirudha/usr/x86_64-apple-darwin12.2.1/lib/libcholmod.so" /Users/mac6-user1/mxe/mxe-octave-anirudha/usr/x86_64-apple-darwin12.2.1/bin/octave

I also had to recompile Octave with a new LDFLAG in order to reserve enough space for changing all dynamic shared library paths. I added this in the src/octave.mk file:
LDFLAGS=-Wl,-headerpad_max_install_names

There are a few other libraries which have this error, and can be fixed by following the above procedure. Finally when all the install names in the faulty libraries/object files are fixed, Octave is ready to be launched.