I'm currently working on migration to Windows Server 2008 and Windows 7 for a customer (don't laugh, it pays my rent). Said customer has a bunch of Solaris boxes, which are used as file servers, running Samba. Obviously, in such a setup you would want to have a single source of login credentials, the Active Directory (with UNIX support added) being a good choice. winbind, a part of Samba, does just that. Now with the existing W2k3 infrastructure, all is fine with the old 3.0 version of Samba. However, Server 2008 moved to NTLMv2, which Samba 3.0 doesn't understand. One could turn off NTLMv2, but this is equivalent to an open invitation to pwn all the boxes, so moving forward to a newer Samba is without alternative.

Ok, so I got myself the latest stable Samba source (3.5.6 at time of writing), spent a day or so installing a gazillion of packages on Solaris to have a working compilation environment, sweared a bit into the general direction of Heimdal, but ended up with a compiled samba. Another day of swearing, removing Solaris-packaged pam_winbind and nss_winbind libraries and replacing them with the freshly-compiled ones, and I was able to join the AD domain. Yay.

Just that it didn't work.

I double-checked nsswitch.conf, smb.conf, krb5.conf, pam.conf etc., but found no mistake. According to the documentation, all was well. But: no login for me.

So I thought I'd move to more familiar territory, and installed a recent Ubuntu server. Stuff actually worked a bit better there. Ok, it keeps forgetting I joined the domain once a day, but other than that, I was able to make it work.

Barely.

I could authenticate and access the smb shares. However, samba would pull a uid out of thin air, instead of using the one stored in AD (which, by the way, does so in a standard-compliant way, using the schema from rfc2307). Ok, more document reading, and I found there was a idmap plugin for AD, which was supposed to do the right thing. I followed the documentation, and lo and behold, domain user login stopped working for good.

Imagine a day of trying to make sense of the Samba logs and frantic googling here.

Finally, I found a clue in the samba.org wiki. There's a section there talking about the use of AD for idmap. And it ends with a comment:

Please note that from 3.0.25 on these values look different as one needs to use the new idmap stuff !
--Schlomo 05:59, 1 May 2007 (CDT)

WTF? This has changed three years ago, and nobody cared to update the documentation?

I finally found a link that helped me solve this issue. Deep inside the bowels of the samba.org web site, in the bug tracker, I found a bug somebody opened who ran into the very same problem I did. There also was a helpful comment by Michael Adam, who provided a working config example. Only two years after Schlomo's note in the wiki above. And more than a year later, the bug (which eventually was decided to be a case of "we should go around and document this stuff"), is still open.

But it gets worse.

After having succeeded on Ubuntu, I tried to port my results to Solaris. Unsuccessfully, as you might have guessed. The initial problem was that the build process didn't bother to compile and install the idmap ad module. That was easy to do manually. However, upon loading this module, Samba would complain about:

Error trying to resolve symbol 'init_samba_module' in /usr/local/samba/lib/idmap/ad.so: ld.so.1: winbindd: fatal: init_samba_module: can't find symbol

Rats.

Next step: I grep for this symbol in the source. Imagine my surprise to find out that this symbol is referenced exactly twice in the entire source code, both calls to dlsym(), related to loading the module. In the 30 or so modules however, not a single trace of this symbol can be found! I considered some black magic generating the symbol on the fly, but inspection of the .so with power tools ("string") confirmed the symbol is just missing.

Ok, so how did it work on Ubuntu, then?

Back to that box, and checking ad.so there. To my mild surprise, I found the symbol in the .so file, unlike on Solaris. Ok, apt-get source samba. And guess what, it downloaded an extensive patch (a whopping 136806 lines), which among other things contains code to add a #define idmap_ad_init init_samba_module line to confdefs.h. And that makes it work.

FFFFUUUUUUUU!!!

To sum it up, Samba documentation has been outdated in important parts, and the code base is in such a sorry state that it doesn't even properly build a single plugin, and all this must have been going on for years now. This implies a lot about the QA that's going on in the Samba project. My guess is somebody forgot to turn off the light at samba.org and tell us it doesn't exist anymore.