LLVM Microconference Notes Welcome to Linux Plumbers Conference 2014. #llvmallthethings Please use this etherpad to take notes. Microconf leaders will be giving a summary of their microconference during the Friday afternoon closing session. Please remember there is no video this year, so your notes are the only record of your microconference. -------------------------------------------------------------------------- Marshall Clow: What's new in the LLVM Ecosystem, and how does that affect the kernel? * A tour of LLVM technologies, concentrating on those of interest to kernel developers. LLVM is now just a name Clang is a compiler based on LLVM Sanitizers: - Runtime reporting of error conditions Address Sanitizer - guard blocks, etc UBSAN - Undefined behavior Memory Sanitizer - Accessing uninialized memory, etc Thread Sanitizer Some but not all Sanitizers in both GCC and Clang Address sanitizer already being used on Linux kernel Use after free, stack overflow, OOB read/write - Uses SLAB allocator only - Using ASAN in GCC ASAN has a performance overhead of 2/1 Replacement for valgrind which is about 10x slower ASAN instrumented binaries can be run in production to catch errors Clang-Tidy - Like LINT on steroids - Can use AST matching, static analyzers - Can detect common errors (arg transposition, etc), invalid parameters, unsafe memory access Bryce has in the past ported kmalloc and kfree to support the checkers New errors can be tried in clang-tidy and if found to be useful can migrate to clang Checkers exist for cyclomatic complexity, other Checkers can suggest fixes (-fix) libclang contains a refactoring framework - uses AST Clang format - could add formatting style for kernel Renato: new Leak Sanitizer - Kostya want to combine into Memory Sanitizer Marshal: UBSAN and ASAN are separate builds -------------------------------------------------------------------------- Behan Webster: LLVMLinux progress * Where do we stand with the patches to support compiling the kernel with clang * Project wiki: http://llvm.linuxfoundation.org/ * Build/test system: git://git.linuxfoundation.org/llvmlinux.git * kernel tree: git://git.linuxfoundation.org/llvmlinux/kernel.git Enabling toolchain choice, allows multiple takes on the code, makes code more portable Working as a collaborative group across companies, individuals Still finding lots of issues, corner cases. The kernel is complex and uncovers many undefined GCC behaviors when building with Clang Clang is very focused on correctness, though also provides GCC compatibility Using GAS and not clang Integrated Assembler, clang still does syntax checking of inline ASM. Can't abuse inline assembly to generate code for external parser. Needs to be valid assembly. IA will likely be used for x86/x86_64 and AArch64. ARM uses pre-unified assembly and is not supported by IA Renato: for invalid assembly, proposed flag for not validating particular ASM, not well received, can kernel fix this? Arnd: Yes we can fix it, no need to hack clang David and Arnd: perhaps we can still abuse the assembler in a syntactically valid way 49 outstanding patches for all arch IA Status - currently turned off for kernel build (toplevel Makefile) JSM: should be set per arch so it can be easily enabled when a particular arch is reay to use IA Arnd: Would like to use IA where possible but would be willing to selectively turn it off per file Behan: about 100 files to instrument IA is faster than using GAS extern inline is still an issue (C99 vs GNU99) Marshal: next version of GCC will default to GNU11 and clang defaults to C11 David: Do we honor regparm attribute? Behan: Not sure? Will follow up http://llvm.org/bugs/show_bug.cgi?id=3997 Renato: Performance of crypto VLAIS patch, is if faster? slower Behan: when we tested last year it was pretty similar code Bloatometer: some better for clang others for GCC something to investigate Arnd: Builtin constant p ? Renato: Unreliable even in GCC -------------------------------------------------------------------------- Jan-Simon Möller: What's left to make allyesconfig on x86_64 * The LLVMLinux is close to be able to compile the full kernel (aka allyesconfig) - what is left to do ? 18 places had to be disabled - now down to ~6 AVX assembly may not be parsed by clang MPILIB - multiple definitions od mpihelp_add Arnd: Is it an extern inline? JSM: possibly, will have to look CONFIG_GCOV_KERNEL - would typically be resolved in Compiler RT Arnd: Don't link against compilerRT, make your own implementations undefined references - is it in the wrong place? Behan: Merge windows tend to cause new breakage David: File bug if AVX is not working. Should be supported JSM: Perhaps flag is just missing RTLIB - undefined references Behan: was fixed for ARM, will look -------------------------------------------------------------------------- Kristof Beyls: LLVM AArch64 Status of AArch64 support in LLVM, and sub projects Upstream LLVM support happened about 2 yr ago 1yr ago for SIMD support Apple open sourced ARM64 backend - merged now with previous AArch64 backend Using LLVM for ARM proprietary compiler - focued on correctness CortexA57 benchmarks - roughly on par (GCC and Clang) GCC maybe slightly ahead - CoreMark, Dhrystone, EEMBC - Clang not optimized for these Status on compiler features: - BigEndian fully supported - Near(?) intrynsics Missing buildbot to check the performance of the code, and what subprojects are working Hard to get LTO working - no LTO linker available that supports AARch64 Gold or LLD will hopefully support AArch64 ILP32 - GCC support For a class of problems it will give you a large improvement in performance Renato: ThreadSanitizer could work for AArch64 but does not work Greg: UBSAN works on AArch64 Libc++ Renato: libunwind support does not support AArch64 - It now does! -------------------------------------------------------------------------- Mark Charlebois: Using LLVM to generate call graphs for the Linux kernel * Build code with clang and get call graphs for free - Interesting: dot files created from clang - can't we create a frontend ? - patch to llvm short, enabled by cflag - frontend created in nodejs (backend in nodejs, presentation in browser) - demo - features colored function by type, - can be huge for the kernel. - force graph view - (js library/framework D3) - code is part of the llvmlinux git tree in tools/KernelViz -------------------------------------------------------------------------- Renato Golin: LLVM ARM Toolchain * a full LLVM toolchain together, from core support tools (like integrated assembler and exception handling) to libraries and tools. What else can we make better? History of ARM LLVM 2010 - present - Compiler is complete - CompilerRT is fully working (libgcc equilavent) - libc++ Renato: libunwind should be in CompilerRt vs libc++ IA turned on by default Focusing on getting the libraries going Still uses libgcc_eh (instead of libunwind) Uses libc++, also works with bionic MCLinker... OpenSource... just doesn't work, and when it does, only partially CompilerRT bot has been recently green Marshal: libc++ tests exist, ABI tests are minimal One final bug fix needed to enable Chromium to build with clang Can thread sanitizer work for ARM? (32 bit)... even with a ship its likely too slow (can hide the actual issues) Snapdragon mcpu flags in LLVM - using cortexA15 The way Android works its beneficial to be able to specify the mcpu - uncertain what to set for Snapdragon -------------------------------------------------------------------------- Bernhard Rosenkraenzer: Building Android userland with clang * Problems when trying to build Android userland with clang, and future steps LLVM Nexus 4 and 5 not quite working, Nexus 7 partially working, Nexus 10 working well GLESv2 wrapper - works using bionic methods to get TLS address, faster assembly implementation fails for clang Init fails when compiled with clang Found bus in MPEG decoder, some QCOM drivers, ... One clang bug (13007) - aligned large number https://android.git.linaro.org/gitweb/android-patchsets.git -------------------------------------------------------------------------- Bernhard Rosenkraenzer: Switching OpenMandriva to clang * Experiences OpenMandriva made while changing the distribution's default compiler to clang 3.5 clang is the default compiler now, and gold is the linker motivated by the rapid improvement of clang 800 build failures - similar ro new gcc version When autoconf fails due to bad flag to clang, hard to debug Nested functions will be removed from RPM X.Org drivers crash the X server - using GCC for now Triplet needed to be fixed, using Fedora style which has armv7hl-linux-gnueabi vs armv7l-linux-gnueabihf Many binary only packages still require libstdc++ so can'y move to libc++ completely ====================================================== Summary: * We discussed the various sanitizers in clang and other tools such as clang-tidy and clang-format * Add kernel specific checkers in clang-tidy * Went over the remaining patches which are required to compile the Linux kernel with clang * The implementation of asm-offsets should be rewritten to not abuse inline assembly (which breaks clang) * The differences in size between some gcc and clang objects need investigating * Per arch ability to enable/disable clang integrated assembler * Follow up to investigate whether clang supports regparm * Investigate outstanding x86 assembly issue and use of extern inline * We need to provide clang specific support for gcov in the kernel * Backend support for AArch64 in clang is complete * Work still needed on llvm linker, other libraries, and validation * The kernel call graph visualization tool should be extended to allow call depth configuration * libunwind needs to be moved to libc++ * Compiler RT can now be cross compiled for ARM * More tests need to be enabled for libc++ * clang compiled Android now mostly works except for requiring gcc for init and GLESv1/2 wrappers * Investigate difference in how code is assmbled for GLESv2 between gas and clang * The OpenMandriva distro is compiled with clang now: * Nested functions will be removed from rpm to support being compiled with clang * Some cross-compile triplets issues need to be fixed * X.Org server crashes when compiled with clang, needs investigation