libusb_free_device_list(, 1) memory leak

[pilatuz]

simple device enumeration method causes memory leak on MAC OS X:

while (1)
{
 libusb_device **list = 0;
 ssize_t N = libusb_get_device_list(NULL, &list);
 printf("%ld\n", long(N));
 libusb_free_device_list(list, 1);
 usleep(10000*(N+1));
}

The MAC OS X version is 10.6.2

Actually, I've tried:

• FTDI D2XX library (which uses libusb-1.0.8)
• libusb (from master branch)
• libusbx (from master branch)

The result is the same: memory leak about 300K per second.
The exactly same code works fine on Linux.

[xiaofan]

Did you call libusb_init() and libusb_exit() pair during the beginning and the end?

Reference:
​http://libusb.6.n5.nabble.com/quot-memory-leak-quot-debug-message-after-libusb-close-and-upon-application-exit-td5581475.html

[xiaofan]

Ah, I see, in your test, libusb_exit() will never be called.

[pilatuz]

I did call libusb_init() just before while(1) loop. And I didn't call libusb_exit() at the end because my program needs to monitor new USB devices continuously.

[hjelmn]

I will try and track this down today. Need to set up a 10.7 virtual machine since valgrind doesn't currently work under 10.8.

[rogueresearch]

You don't need valgrind to find simple leaks. Just use Instruments or 'leaks' (see 'man leaks').

[hjelmn]

I ran valgrind on the reproducer and it found 0 definitely or possibly lost blocks. It did however find still reachable memory from usb_get_next_device calling IOCreatePlugInInterfaceForService and QueryInterface?. Now, there are a couple of bugs in usb_get_next_device that are fixed in my git tree but they do not fix the "leak". As far as I can tell there is nothing wrong in libusb but there may be something wrong in the kernel. I am working on a workaround that 1) generates an internal device list on the first call to libusb_init, 2) only updates this device list when a device is either plugged in or unplugged (this will involve writing some of the hotplug code I need to write anyway), and 3) simply copies the internal device list when libusb_get_device_list is called.

BTW, I did confirm that amount of "still reachable" memory is correlated to the number of times libusb_get_device_list is called in the reproducer.

[hjelmn]

I thought I would give some additional info. Using leaks under 10.6.7 on a 2011 MBP shows a growing number of allocated blocks when running your reproducer with libusb 1.0.9 (though it doesn't show any "leaked" memory). With 10.8.1 and 10.8.2 I can not reproduce the problem on either a rMBP or the 2011 MBP.

More evidence this is Apple's problem not libusb's. I don't know when they fixed the bug but it is fixed by 10.8.1. I will try with 10.7.x tomorrow and see if it exists there as well.

That said, I have a workaround that *may* make its way into 1.0.10. The fix involves writing some of the hotplug support that is needed for 1.1.x.

[xiaofan]

Not so sure if I am doing the right thing but it seems there is no leak under Mac OS X Lion 10.7.5.

mymacmini:test xiaofanc$ cat testleak.c
#include <stdio.h>
#include <libusb-1.0/libusb.h>

int main(void)
{

libusb_init(NULL);
while (1){

libusb_device list = 0;
ssize_t N = libusb_get_device_list(NULL, &list);
printf("%ld\n",N);
libusb_free_device_list(list, 1);
usleep(10000*(N+1));

}

}

mymacmini:pkgconfig xiaofanc$ leaks testleak
Process: testleak [13962]
Path: /Users/xiaofanc/work/libusb/darwin_test/test/testleak
Load Address: 0x101aaa000
Identifier: testleak
Version: ??? (???)
Code Type: X86-64 (Native)
Parent Process: bash [1875]

Date/Time?: 2012-10-04 10:45:02.920 +0800
OS Version: Mac OS X 10.7.5 (11G56)
Report Version: 7

leaks Report Version: 2.0
Process 13962: 791 nodes malloced for 104 KB
Process 13962: 0 leaks for 0 total leaked bytes.

[hjelmn]

Yup, that should reproduce the "leak" if it exists. If you leave the reproducer running and call leaks periodically you will see the number of nodes increase. This happens under 10.6.x but not under 10.8.x (or 10.7.x it seems). I think it is clear this bug is in IOKit (from 10.6.x) and not libusb.

The next release of libusb will change how devices are enumerated by the darwin backend. The change should reduce the amount of memory leaked by IOKit (fewer calls to the leaky function) and eliminate the leak on repeated calls to libusb_get_device_list.

I am going to mark this ticket as invalid.