A while ago I spend some time in the osx heap allocator. Man there is so much crap in there, unbelieable.
One of the things I ran across was that their realloc implementation will never resize to something smaller. While technically this isn’t a violation of the c standard it’s still pretty retarded.
One other surprising thing I found was that their calloc code in older versions fuckedup boundschecking for 64-bit: http://cvs.opendarwin.org/cgi-bin/cvsweb.cgi/Libc/gen/malloc.c?rev=1.1.1.1&content-type=text/x-cvsweb-markup&cvsroot=apple
#define MAX_ALLOCATION 0xc0000000 // beyond this, assume a programming error
…
void *malloc_zone_calloc(malloc_zone_t *zone, size_t num_items, size_t size) {
void *ptr;
if (malloc_check_start && (malloc_check_counter++ >= malloc_check_start)) {
internal_check();
}
if (((unsigned)num_items >= MAX_ALLOCATION) || ((unsigned)size >= MAX_ALLOCATION) || ((long long)size * num_items >= (long long) MAX_ALLOCATION)) {
/* Probably a programming error */
fprintf(stderr, “*** malloc_zone_calloc[%d]: arguments too large: %d,%d\n”, getpid(), (unsigned)num_items, (unsigned)size);
return NULL;
}
ptr = zone->calloc(zone, num_items, size);
if (malloc_logger) malloc_logger(MALLOC_LOG_TYPE_ALLOCATE | MALLOC_LOG_TYPE_HAS_ZONE | MALLOC_LOG_TYPE_CLEARED, (unsigned)zone, num_items * size, 0, (unsigned)ptr, 0);
return ptr;
}
Incase you don’t spot the bug here, size_t is 64 bit (on 64-bit machines), but a cast to unsigned means unsigned int which is 32 bits long !!! hence there can be truncation later on !
in new versions they just removed the boundscheck, I kid you not, you just can’t make this stuff up.
In the older versions malloc and the like had the same flawed lengthcheck which doesn’t make sense at all !!!! you shouldn’t do boundschecking on the length at this level. the app that uses malloc should do it. The only thing you might want to check for is int overflow. Speaking of this limitation, the OpenBSD kernel memory allocator has very simular behavior, but only even more retarded:
void *
malloc(unsigned long size, int type, int flags)
{
struct kmembuckets *kbp;
struct kmemusage *kup;
struct freelist *freep;
long indx, npg, allocsize;
int s;
caddr_t va, cp, savedlist;
… some debug crap …
if (size > 65535 * PAGE_SIZE)
panic(”malloc: allocation too large”);
…
}
for those who can’t read c code, what this means is, if you tell malloc to allocate a really big chunck it will instead kernel panic. bug or feature ? I guess it was ok to do this in 1975, but It’s 2006 for fucks sake. ofcourse netbsd suffers from the same bug/feature. Not too long ago you could trigger this from the ELF loading code and I believe there are still some obsd drivers that allow you to trigger it.
In light of the whole calloc fuckup I started doing some googling, since I remember an advisory being published about this several years ago. http://cert.uni-stuttgart.de/advisories/calloc.php. Someone made the connection with fread and fwrite and it turns out they also contain a simular int overflow. since the osx guys stole fread/fwrite from FreeBSD, I checked fbsd and it appeared to have this int overflow. Then I figured I’d check openbsd which also has this int overflow. What’s somewhat more worrysome is that obsd’s fread and fwrite are NOT thread-safe especially since obsd recently got a whole new threading implementation: http://www.openbsd.org/papers/eurobsd2005/tedu-rthreads.pdf. I figured I’d check the obsd manpage for fread and fwrite and see if they mention that it’s not thread-safe. but they didn’t, infact, their fread and fwrite manpage hasn’t been update since 1994 !!!! It should come as no surprise that netbsd has the same int overflow issue. I didn’t even try to check glibc, I still want to hold on to my sanity for a while :)
SMALL UPDATE: There is some discussion on the netbsd current-users mailing list about this. Some seem to have misinterpreted my posting. The osx and NetBSD userland allocator have pretty much nothing in common and I wasn’t hinting towards that. But the (older version of) osx userland allocator and NetBSD/OpenBSD kernel allocator have 1 thing in common, namely a check like
if (size > something) bailout;
which imo doesn’t make sense at all especially if “bailout” is a kernel panic ! The other thing I mentioned about NetBSD was an int overflow in their fread and fwrite implementation. I made no other statements about NetBSD.