How to submit an LLVM bug report

Introduction - Got bugs?

If you’re working with LLVM and encounter a bug, we definitely want to know about it. This document describes what you can do to increase the odds of getting it fixed quickly.

🔒 If you believe that the bug is security related, please follow How to report a security issue?. 🔒

Basically, you have to do two things at a minimum. First, decide whether the bug crashes the compiler or if the compiler is miscompiling the program (i.e., the compiler successfully produces an executable, but it doesn’t run right). Based on what type of bug it is, follow the instructions in the linked section to narrow down the bug so that the person who fixes it will be able to find the problem more easily.

Once you have a reduced test case, go to the LLVM Bug Tracking System and fill out the form with the necessary details (note that you don’t need to pick a label, just use if you’re not sure). The bug description should contain the following information:

  • All information necessary to reproduce the problem.

  • The reduced test case that triggers the bug.

  • The location where you obtained LLVM (if not from our Git repository).

Thanks for helping us make LLVM better!

Crashing Bugs

More often than not, bugs in the compiler cause it to crash—often due to an assertion failure of some sort. The most important piece of the puzzle is to figure out if it is crashing in the Clang front-end or if it is one of the LLVM libraries (e.g., the optimizer or code generator) that has problems.

To identify the crashing component (the front-end, middle-end optimizer, or backend code generator), run the clang command line as you were when the crash occurred, but with the following extra command line options:

  • -emit-llvm -Xclang -disable-llvm-passes: If clang still crashes when passed these options (which disable the optimizer and code generator), then the crash is in the front-end. Jump ahead to front-end bugs.

  • -emit-llvm: If clang crashes with this option (which disables the code generator), you’ve found a middle-end optimizer bug. Jump ahead to middle-end bugs.

  • Otherwise, you have a backend code generator crash. Jump ahead to code generator bugs.

Front-end bugs

On a clang crash, the compiler will dump a preprocessed file and a script to replay the clang command. For example, you should see something like

PLEASE ATTACH THE FOLLOWING FILES TO THE BUG REPORT:
Preprocessed source(s) and associated run script(s) are located at:
clang: note: diagnostic msg: /tmp/foo-xxxxxx.c
clang: note: diagnostic msg: /tmp/foo-xxxxxx.sh

The creduce tool helps to reduce the preprocessed file down to the smallest amount of code that still replicates the problem. You’re encouraged to use creduce to reduce the code to make the developers’ lives easier. The clang/utils/creduce-clang-crash.py script can be used on the files that clang dumps to help with automating creating a test to check for the compiler crash.

cvise is an alternative to creduce.

Middle-end optimization bugs

If you find that a bug crashes in the optimizer, compile your test-case to a .bc file by passing “-emit-llvm -O1 -Xclang -disable-llvm-passes -c -o foo.bc”. The -O1 is important because -O0 adds the optnone function attribute to all functions and many passes don’t run on optnone functions. Then run:

opt -O3 foo.bc -disable-output

If this doesn’t crash, please follow the instructions for a front-end bug.

If this does crash, then you can debug this with the llvm-reduce tool. Create a script that reproduces the crash and run:

llvm-reduce --test=path/to/script foo.bc

which should produce reduced IR that reproduces the crash.

Tip

llvm-reduce -j $NUM_THREADS is multi-threaded and can therefore potentially be much faster.

Tip

Reduction is fastest and most effective the simpler the reproduction script is. Ideally, this will be running opt with a single pass. The most effective way to extract the IR before a specific point is a two step process. First, run the testcase with the -print-pass-numbers flag. This will print the name of a pass and an integer ID. You can then use the last ID printed before the crash and add 3 flags, -print-before-pass-number=<integer-id> -print-module-scope -ir-dump-directory=/my/debug/path. This will place failing IR files in the given directory. -print-before-pass-number is the minimum required flag, but will not produce an output directly consumable by a tool. It will print to stderr, and will be incomplete in most situations without -print-module-scope.

A more brute force approach is to use the --print-before-all --print-module-scope flags to dump the IR before every pass. Be warned that this is very verbose.

Backend code generator bugs

If you find a bug that crashes clang in the code generator, compile your source file to a .bc file by passing “-emit-llvm -c -o foo.bc” to clang (in addition to the options you already pass). Once you have foo.bc, one of the following commands should fail:

  1. llc foo.bc

  2. llc foo.bc -relocation-model=pic

  3. llc foo.bc -relocation-model=static

If none of these crash, please follow the instructions for a front-end bug. If one of these crashes, you should be able to reduce this with llvm-reduce, similar to middle end bugs. In this case, your test script should use llc instead of opt.

Please run this, then file a bug with the instructions and reduced .bc file that llvm-reduce emits. If something goes wrong with llvm-reduce, please submit the foo.bc file and the option that llc crashes with.

LTO bugs

If you encounter a bug that leads to crashes in the LLVM LTO phase when using the -flto option, follow these steps to diagnose and report the issue:

Compile your source file to a .bc (Bitcode) file with the following options, in addition to your existing compilation options:

export CFLAGS="-flto -fuse-ld=lld" CXXFLAGS="-flto -fuse-ld=lld" LDFLAGS="-Wl,-plugin-opt=save-temps"

These options enable LTO and save temporary files generated during compilation for later analysis.

On Windows, use lld-link as the linker. Adjust your compilation flags as follows: * Add /lldsavetemps to the linker flags. * When linking from the compiler driver, add /link /lldsavetemps in order to forward that flag to the linker.

Using the specified flags will generate four intermediate bytecode files:

  1. a.out.0.0.preopt.bc (Before any link-time optimizations (LTO) are applied)

  2. a.out.0.2.internalize.bc (After initial optimizations are applied)

  3. a.out.0.4.opt.bc (After an extensive set of optimizations)

  4. a.out.0.5.precodegen.bc (After LTO but before translating into machine code)

Execute one of the following commands to identify the source of the problem:

  1. opt "-passes=lto<O3>" a.out.0.2.internalize.bc

  2. llc a.out.0.5.precodegen.bc

If one of these do crash, you should be able to reduce this with llvm-reduce command line (use the bc file corresponding to the command above that failed):

llvm-reduce --test reduce.sh a.out.0.2.internalize.bc

Example of reduce.sh script

$ cat reduce.sh
#!/bin/bash -e

path/to/not --crash path/to/opt "-passes=lto<O3>" $1 -o temp.bc  2> err.log
grep -q "It->second == &Insn" err.log

Here we have grepped for the failed assert message.

Please run this, then file a bug with the instructions and reduced .bc file that llvm-reduce emits.

Miscompilations

If clang successfully produces an executable, but that executable doesn’t run right, this is either a bug in the code or a bug in the compiler. The first thing to check is to make sure it is not using undefined behavior (e.g., reading a variable before it is defined). In particular, check to see if the program is clean under various sanitizers (e.g., clang -fsanitize=undefined,address) and valgrind. Many “LLVM bugs” that we have chased down ended up being bugs in the program being compiled, not LLVM.

Once you determine that the program itself is not buggy, you should choose which code generator you wish to compile the program with (e.g., LLC or the JIT) and optionally a series of LLVM passes to run. For example:

bugpoint -run-llc [... optzn passes ...] file-to-test.bc --args -- [program arguments]

bugpoint will try to narrow down your list of passes to the one pass that causes an error, and simplify the bitcode file as much as it can to assist you. It will print a message letting you know how to reproduce the resulting error.

The OptBisect page shows an alternative method for finding incorrect optimization passes.