Reducing the delta with upstream version of sanitizers

I've been actively working on reducing the delta with the local copy of sanitizers with upstream LLVM sources. Their diff has been reduced to less than 2000 Lines Of Code. I've pushed to review almost all of the local code and I'm working on addressing comments from upstream developers.

LLVM changes

The majority of work was related to interceptors. There was a need to cleanup the local code and develop dedicated tests for new interceptors whenever applicable (i.e. always unless this is a syscall modifying the kernel state such as inserting kernel modules).

Detailed list of commits merged with the upstream LLVM compiler-rt repository:

• Split getpwent and fgetgrent functions in interceptors
• Try to unbreak the build of sanitizers on !NetBSD
• Disable recursive interception for tzset in MSan
• Follow Windows' approach for NetBSD in AlarmCallback()
• Disable XRay test fork_basic_logging for NetBSD
• Prioritize the constructor call of __local_xray_dyninit() (investigated with help of Michal Gorny)
• Adapt UBSan integer truncation tests to NetBSD
• Split remquol() from INIT_REMQUO
• Split lgammal() from INIT_LGAMMAL
• Correct atexit(3) support in MSan/NetBSD (with help of Michal Gorny investigating the failure on Linux)
• Add new interceptor for getmntinfo(3) from NetBSD
• Add new interceptor for mi_vector_hash(3)
• Cast _Unwind_GetIP() and _Unwind_GetRegionStart() to uintptr_t
• Cast the 2nd argument of _Unwind_SetIP() to _Unwind_Ptr (reverted as it broke "MacPro Late 2013")
• Add interceptor for the setvbuf(3) from NetBSD
• Add a new interceptor for getvfsstat(2) from NetBSD

A single patch landed in the LLVM source tree:

• Swap order of discovering of -ltinfo and -lterminfo (originated by Ryo Onodera in pkgsrc)

Patches submitted upstream and still in review:

• Add interceptors for the sha1(3) from NetBSD
• Add interceptors for the md4(3) from NetBSD
• Add interceptors for the rmd160(3) from NetBSD
• Add interceptors for md5(3) from NetBSD
• Add a new interceptor for nl_langinfo(3) from NetBSD
• Add a new interceptor for fparseln(3) from NetBSD
• Add a new interceptor for modctl(2) from NetBSD
• Add a new interceptors for statvfs1(2) and fstatvfs1(2) from NetBSD
• Add a new interceptors for cdbr(3) and cdbw(3) API from NetBSD
• Add interceptors for the sysctl(3) API family from NetBSD
• Add interceptors for the fts(3) API family from NetBSD
• Implement getpeername(2) interceptor
• Add new interceptors for vis(3) API in NetBSD
• Add new interceptor for regex(3) in NetBSD
• Add new interceptor for strtonum(3)
• Add interceptors for the strtoi(3)/strtou(3) from NetBSD
• Add interceptors for the sha2(3) from NetBSD

Patches still kept locally:

• ASan thread's termination destructor
• MSan thread's termination destructor
• Interceptors for getchar(3) API (might be abandoned as FILE/DIR sanitization isn't done)
• Incomplete interceptor for mount(2) (might be abandoned as unfinished)

This month I've received also a piece of help from Michal Gorny who improved the NetBSD support in LLVM projects with the following changes:

• [unittest] Skip W+X MappedMemoryTests when MPROTECT is enabled
• [cmake] Fix detecting terminfo library

Changes to the NetBSD distribution

I've reduced the number of changes to the src/ distribution to corrections related to interceptors.

• Document SHA1FileChunk(3) in sha1(3)
• Fix link sha1.3 <- SHA1File.3
• Define MD4_DIGEST_STRING_LENGTH in <md4.h>
• Correct the documentation of cdbr_open_mem(3)

Plan for the next milestone

I will keep upstreaming local LLVM patches (less than 2000LOC to go!).

This work was sponsored by The NetBSD Foundation.

The NetBSD Foundation is a non-profit organization and welcomes any donations to help us continue funding projects and services to the open-source community. Please consider visiting the following URL, and chip in what you can:

http://netbsd.org/donations/#how-to-donate [0 comments]