NetBSD LLVM Sanitizers in The Bay Area

I have presented the state of NetBSD sanitizers during two conferences in the San Francisco Bay Area: Google Summer of Code Mentor Summit (Mountain View) and MeetBSDCa (Santa Clara, Intel Campus SC12). I've also made progress in upstreaming of our local patches to LLVM sanitizers and introducing generic NetBSD enhancements there.

The Bay Area

I took part (together with William Coldwell - cryo@) in the GSOC Mentor Summit as a NetBSD delegate. I've presented during the event a presentation with a quick introduction to NetBSD, track history of GSoC involvement and the LLVM Sanitizers work with a stress of sanitizers.

• NetBSD and userland & kernel SANITIZERS

The MeetBSDCa conference is a continuation of the MeetBSD conferences from Poland. I took part there as a speaker talking about Userland Sanitizers in NetBSD. I've also presented the state of virtualization in NetBSD during a discussion panel. Additionally I've prepared a lightning talk about NetBSD Kernel sanitizers and quick status update from The NetBSD Foundation. Unfortunately the schedule was last minute changed (introduction of BSD history talk in the slot of lightning presentations) and the closing ceremony had different proceeding. Nonetheless, I'm sharing these additional quick presentations.

• Bug detecting software in the NetBSD userland: MKSANITIZER
• NetBSD kernel sanitizers
• MeetBSDCa 2018: The NetBSD Foundation update

During the former conference it was a great opportunity to meet people from other Open Source projects, a lot of them are in interaction with NetBSD developers during the process of upstreaming local support patches. During the latter conference it was an opportunity to meet BSD people and people closer to hardware companies.

Upstreaming process of LLVM Sanitizers

I've upstreamed a number of patches to the LLVM source tree. The changes can be summarized as:

• Further reworking the code and approaching the state of installing of sysctl*() inteceptors.
• Fixing or marking failing or hanging tests in the sanitizer test-suites.
• Adapting definitions of syscalls and ioctl(2) operations for NetBSD 8.99.25.

Detailed list of commits merged with the upstream LLVM compiler-rt repository:

• Update ioctl(2) operations for NetBSD 8.99.25
• Update generate_netbsd_ioctls.awk for NetBSD 8.99.25
• Diable test suppressions-library for NetBSD/i386
• Disable BufferOverflowAfterManyFrees for NetBSD
• Mark breaking asan tests on NetBSD
• Switch getline_nohang from XFAIL to UNSUPPORTED for NetBSD
• Mark vptr-non-unique-typeinfo as a broken test for NetBSD/i386
• Mark breaking sanitizer_common tests on NetBSD
• Handle NetBSD alias for pthread_sigmask
• Cast the return value of _Unwind_GetIP() to uptr
• Mark interception_failure_test with XFAIL for NetBSD
• Disable ASan test asan_and_llvm_coverage_test for NetBSD
• Adapt ASan test heavy_uar_test for NetBSD
• Mark breaking TSan tests on NetBSD with XFAIL
• Cleanup includes in sanitizer_platform_limits_netbsd.cc
• Regenerate syscall hooks for NetBSD 8.99.25
• Update generate_netbsd_syscalls.awk for NetBSD 8.99.25
• Handle pthread_sigmask in DemangleFunctionName()
• Drop now hidden ioctl(2) operations for NetBSD
• Handle NetBSD symbol mangling for tzset
• Handle NetBSD symbol mangling for nanosleep and vfork
• Mark test/tsan/getline_nohang as XFAIL for NetBSD
• Disable the GNU strerror_r TSan test for NetBSD
• Mark test/tsan/ignore_lib5 as unsupported for NetBSD
• Mark intercept-rethrow-exception.cc as XFAIL on NetBSD
• Disable failing tests lib/asan/tests on NetBSD
• Skip unsupported MSan tests on NetBSD
• Mark 4 MSan tests as XFAIL for NetBSD
• Mark MSan fork test as UNSUPPORTED on NetBSD
• Reflect the current reality and disable lsan tests on NetBSD
• Use PTHREAD_STACK_MIN conditionally in a test
• Remove remnant code of using indirect syscall on NetBSD
• Don't harcode -ldl test/sanitizer_common/TestCases
• Disable TestCases/pthread_mutexattr_get on NetBSD
• Fix Posix/devname_r for NetBSD
• Unwind local macro DEFINE_INTERNAL()
• Introduce internal_sysctlbyname in place of sysctlbyname

Frequently asked question

People keep asking me about rationale of some design decisions in sanitizers and whether something could be done better. One of such places is to reuse more of libc internals and to not keep bypassing it whenever possible. The motivation to keep redoing the same work for NetBSD is to keep close to the upstream (mostly Linux & Android) source code with a minimal delta between NetBSD vs others support. Doing some operations in a more convenient way is tempting, but it's a danger that someone will need to keep maintaining a larger diff, especially since upstream developers will focus on their own OSes rather than trying to adapt their patches for potentially alternative approaches.

Plan for the next milestone

I will keep upstreaming local LLVM patches (almost 2500LOC to go!).

This work was sponsored by The NetBSD Foundation.

The NetBSD Foundation is a non-profit organization and welcomes any donations to help us continue funding projects and services to the open-source community. Please consider visiting the following URL, and chip in what you can:

http://netbsd.org/donations/#how-to-donate [1 comment]