Update: Howto compile the 2.6.23 kernel for the Mac Pro

I've updated my previous post with a new .config file.

I had some problems with my wacom tablet (an IRQ conflict after about 10 minutes usage). The reason was that i still had legacy IDE support compiled statically into the kernel and that is not necessary (or even good at all) for the Mac Pro. So it's removed now in the new .config.

Posted byvfxdude at 10:39 AM  

19 comments:

macaholic said... November 17, 2007 at 10:44 PM  

Hi, I noticed in your first post you mentioned you have 4gbs of ram. I just installed 2 more gbs, for a total of 3, and ubuntu (7.10) is only seeing 2 of them. Did you have to do anything special to make them work, and is all you memory installed on the same chip?

vfxdude said... November 17, 2007 at 10:53 PM  

Hi macaholic,

actually since than I upgraded to 8GB. That's 8 times 1GB, so all the slots are filled now. Nothing needs to be set in Ubuntu to be able to see more than 2GB.

Did you try to boot into OS X and see if it shows only 2GB as well?

macaholic said... November 18, 2007 at 5:18 AM  

Yes, in OSX the 3gbs show up fine, it is just in Ubuntu :(

vfxdude said... November 18, 2007 at 9:51 AM  

Two things:

1. Make sure you use the 64-but version of Ubuntu.

2. Compile the custom kernel as mentioned in this post.

This should fix your problem.

Let me know how it goes!

macaholic said... November 18, 2007 at 6:41 PM  

So the only way I can make it work is if I wipe my current install, install the 64 bit version, and then compile my own kernel?

vfxdude said... November 18, 2007 at 8:09 PM  

I'm assuming you're running a Mac Pro. In that case your processor supports 64-bit. I'd recommend you to install the 64-bit version of Ubuntu anyway.

This alone might fix the problem.

If not, go ahead and compile the custom kernel.

And please make sure to report back how it went. Thanks!

pixelcowboy said... February 17, 2008 at 12:25 PM  

Hey man, you haven't posted in a while. I was just wondering how has the experience gone so far and if it is actually worth it... I also work in vfx and would like to run Houdini/Nuke/Maya etc in a similar setup.

vfxdude said... February 17, 2008 at 1:29 PM  

Hi pixelcowboy,

I've been using Nuke with my setup (Ubuntu 7.10 64-bit, custom 2.6.23 kernel, on a Mac Pro) in production for about 2 months now. It works great for me. The performance is amazing. No way I'll go back to OS X or Windows. ;)

Just make sure you have an Nvidia graphics card...

As far as Houdini goes, I've only been using the Apprentice version for some tutorials so I can't tell you how it is in production but what I've seen so far is that it's very stable and fast.

Cheers

pixelcowboy said... February 17, 2008 at 1:56 PM  

That's really awesome man, and by the way, any networking issues that might be noted? Do you have any special network setup that required further tweaking?
Also, did you manage to fix your wacom problem completely?
Thanks again for the help dude

vfxdude said... February 17, 2008 at 2:28 PM  

Yep, wacom is working fine.

Networking was set up by a network specialist (as we have a rather complicated NFS-mounted home directory setup). But even that was no problem, so I'm sure simpler setups shouldn't give you any trouble...

pixelcowboy said... February 17, 2008 at 3:57 PM  

Last question if I may, I hope I'm not too annoying. I love ubuntu, but I'm having difficutly convincing people to adopt it in a production environment (they see it as a desktop distribution). Is there any technical advantage in particular why you chose it over other distributions?

andy said... February 24, 2008 at 2:35 AM  

Hi there, fellow VFX guy here...wondering how it is going with your setup? has the fan problem subsided? thanks for your great blog! -a

vfxdude said... February 24, 2008 at 1:43 PM  

pixelcowboy:
I wanted a Debian-based distribution because of the package manager and Ubuntu has a huge community so there's always someone with the same problem as you. And the packages are always quiet up-to-date (as opposed to Debian where the release cycle is really long). The other thing is that I prefer Gnome to KDE.

There seems to be a lot of prejudice against Ubuntu in a production environment (I have to admit that I was sceptical as well - and all the IT guys too who I told about it). But we all agree now that it was the right distribution for us.

andy: As of 2.6.23 the fan problem is still there. I wrote a Python script that checks the temperature every second and sets the RPM accordingly. Works great! I haven't tried 2.6.24 yet, it's supposed to have all the patches in there already but I don't if it addresses the fan problem. I'll have to check it out in the next couple of weeks.

andy said... February 25, 2008 at 2:15 AM  

thanks VFXdude, I tried to install yesterday the latest ubuntu on a 2nd drive but kept on getting to a blank screen. It would come up on the ubuntu install, then after clicking to "start or install", it would just start the kernel, then hang...

Is this similar to the problem you were having with the splash screen?

I'm on a 2008 macpro, with an 8800gt

ps: the new nuke5 looks awesome!

CGwynne said... April 11, 2008 at 10:06 AM  

Hi VFXdude, thanks for all the info you've posted. I've searched the net for three days trying to figure out how to get my brand new 8core 3.2 16GB Mac Pro running Linux and have just gotten nowhere. Your guide to install still didn't solve my problem of getting to a black screen in Gutsy x64 before I could even click on install in the GUI, but I just cloned a drive from another working box and it booted right up with the Nvidia driver already going.

But I'm about to do the custom kernel as you outline (I haven't been on Linux in about 5 years so I'm really rusty) but I'm concerned about the fan issue you mentioned since I'm running some fluid sims and will max out the cpus and ram usage. Could you post the python script for the fan issue since my box is only three days old, I've been through hell to get it and I don't want to kill it yet.

Thanks.

vfxdude said... June 14, 2008 at 6:54 PM  

Hey CGwynne,

sorry for my late post, haven't checked the comments for a while, hope you figured it out meanwhile.

I'd rather not post the source, coz it's a huge hack and I have no idea if it works on other configurations without glitches. But here are a few hints to get you started:

1. You need the binary called "coretemp" to read the core temperature. This used to be part of the mactel-patches, but somehow I can't find it in there anymore...

2. You can set the fan speed, by first setting the fan to manual control. Just write the value 1 into the following file:

/sys/devices/platform/applesmc.768/fan1_manual (that's for fan nr. 1)

Than write the RPM into the following file:

/sys/devices/platform/applesmc.768/fan1_output (again, for fan nr. 1)

And you have to do that for all the fans.

Hope this helped!

Cheers

Simen said... July 17, 2008 at 4:00 PM  

How is the Python script comming? Maybe making it open source and put it on the web? I think alot of people want this script to keep their Mac quite, secure and cool :)

vfxdude said... August 21, 2008 at 1:20 PM  

Hi, I'm working to release a version of the script within the next two weeks (as I finally got a few days off). Thanks for your patience!

Simen said... September 8, 2008 at 8:46 PM  

Is it comming along?

Post a Comment

FeedBurner FeedCount

Subscribe to RSS feed