LuaJIT Source Code Analysis (Part 1): Setting Up the Debugging Environment

As is widely known, LuaJIT is a Just-In-Time Compiler (JIT) for the Lua programming language, and it executes Lua code at an impressive speed. However, resources about LuaJIT source code are quite sparse online, so I have to dig in and start reading from scratch.

Starting anything is always the hardest part. To analyze the source code, the first step is to set up an environment where you can debug your code. Unfortunately, even the initial step has limited references available online, maybe due to the complexity of compilation process.

First, check out the source code. The official Git repository is as follows:

git clone https://luajit.org/git/luajit.git

There is also a mirror link on GitHub.

We plan to use Visual Studio for debugging. LuaJIT provides a script for MSVC compilation called msvcbuild.bat, located in the src directory. By examining this script, we can see the compilation process is divided into three parts. First, it builds a minilua, which is a subset of Lua native code used to determine whether the target platform is 32-bit or 64-bit and to execute Lua scripts for generating platform-specific instructions. Next is buildvm, which generates mappings for various library functions. Finally, it compiles various Lua libraries and produces the final LuaJIT. This script should be executed in the Visual Studio Command Prompt environment. According to the comments from the script, there are four additional compilation options:

@rem Open a "Visual Studio Command Prompt" (either x86 or x64).
@rem Then cd to this directory and run this script. Use the following
@rem options (in order), if needed. The default is a dynamic release build.
@rem
@rem   nogc64   disable LJ_GC64 mode for x64
@rem   debug    emit debug symbols
@rem   amalg    amalgamated build
@rem   static   static linkage

We don’t need to include these options for our compilation. Additionally, this script performs some cleanup tasks at the end of the build process, removing intermediate codes and compiled objs. Since we want to keep these files for debugging, you may need to comment out this cleanup logic in the script.

@REM @del *.obj *.manifest minilua.exe buildvm.exe
@REM @del host\buildvm_arch.h
@REM @del lj_bcdef.h lj_ffdef.h lj_libdef.h lj_recdef.h lj_folddef.h

For example, in Visual Studio 2022, we first locate the 64-bit Command Prompt:

Click on it, then cd to the LuaJIT src directory and run the compilation script msvcbuild.bat:

The compilation is quick. If everything goes smoothly, you will see a success message in the output at the end.

In the src directory, you’ll also find the built luajit.exe. Double-click it to run; this is our final Lua virtual machine.

Next, create a new directory at the same level as the src directory to store the Visual Studio project. Open Visual Studio, create a new empty console C++ project, and then add header files and source files from the src directory to the project.

If we press F5 to debug now, we might encounter an error.

error C4996: ‘strerror’: This function or variable may be unsafe. Consider using strerror_s instead. To disable deprecation, use _CRT_SECURE_NO_WARNINGS. See online help for details.

This is because LuaJIT source code uses the strerror C API, which is considered unsafe, and the compiler recommends using strerror_s instead. To avoid modifying the source code, we just simply add the _CRT_SECURE_NO_WARNINGS macro to the project.

Try again, the compilation may pass, but you might encounter another error during the linking stage.

The error indicates that some functions are defined multiple times, all of them are from ljamalg.obj. So check the corresponding source file, ljamalg.c, and we find that it includes several source files directly.

Back to msvcbuild.bat, you’ll see that ljamalg.c is only compiled when the amalgamation option is enabled.

@if "%1"=="amalg" goto :AMALGDLL
...
:AMALGDLL
%LJCOMPILE% %LJDYNBUILD% ljamalg.c

Since we’re not using the amalgamation option, ljamalg.c should be excluded from the VS project. The purpose of the amalgamation option is explained on the LuaJIT website.

The build system has a special target for an amalgamated build, i.e. make amalg. This compiles the LuaJIT core as one huge C file and allows GCC to generate faster and shorter code. Alas, this requires lots of memory during the build. This may be a problem for some users, that’s why it’s not enabled by default. But it shouldn’t be a problem for most build farms. It’s recommended that binary distributions use this target for their LuaJIT builds.

In short, this option compiles the LuaJIT core into a single large C file, allowing the compiler to generate faster and shorter code.

After removing ljamalg.c, let’s try again, but unfortunately, the linking might still fail.

Recalling the LuaJIT build process, the mappings of various library functions in the VM are handled by buildvm. They don’t have corresponding source code but their generated object file is located in the src directory, which is also managed by the msvcbuild.bat script.

buildvm -m peobj -o lj_vm.obj

This object file should contain the various functions that need to be referenced. So, manually add this to the VS project.

Try again, now LuaJIT should finally start successfully. By default, when debugging, no parameters are provided, so it will receive input from stdin.

Add a breakpoint at the entry point of the main function, and then press F5 again. We should be able to debug LuaJIT successfully.

Great! Now we can happily delve into the LuaJIT source code.