OpenWRT, RAID Crashes, and a Truck

My first day at my host org, I arrived minutes before the moving truck. My organization got new digs and I was just in time to help set up. On Weds of that week, we had the annual staff retreat out on Watauga Lake, and I got a good introduction to all the staff and various projects going on here. As an environmental nonprofit, much of our work is the type of advocacy campaign that VISTA wants me to step carefully around. My project is more of an education program -- teaching landowners about sustainable forestry techniques -- so that's my main focus.

Early in my second week, I sat down with several folks in the organization to get an in-depth feel for the sustainable forestry project and it's future. We're planning a forum for landowners and forestry professionals, as both an online community and an information resource. Much of the initial planning is done, but I get to carry the ball into the endzone. This means working with the project partners, finding and arranging hosting, working with a professional web design company, developing initial content, and recruiting others to continue the content ....seems like most other aspects of the project (hopefully, with the exception of fundraising!). Our in-house meeting went well and I'm arranging a conference call to introduce myself to our project partners.

I'm also working on maintaining and extending their technical capacity, by assessing their needs and expanding their current capabilities. I've already extended their LAN and WirelessLAN capabilities with an older Linksys WiFi router (I *heart* OpenWRT!). The WiFi router provided by our DSL supplier, it turns out, doesn't play nicely with newer MacBooks.

I'm working through an issue with their fileserver (minor things, ya know, like replacing a drive in the main RAID set that decided that Friday afternoon was the perfect time to die). One drive dropped out of the RAID mirror, and I got the local volunteer who built the server to stop by and help me kick off a rebuild. Most of the time this works fine and things are back to normal (the rebuild takes forever, but you can still access the existing drive). Unfortunately, around 3pm last Friday I realized the network drive wasn't accessible, and sure enough the rebuild had encountered a serious problem with the offline drive (think "click of death"). I consulted our volunteer, rebooted the server, and waited until everyone was done for the day before trying the rebuild again. Sure enough, the problem drive started making its click of death again, and the server locked up. Now we're up and running, but without a mirror, and waiting for a replacement drive to come in.

I'm also rolling out an intranet webserver so I can evaluate and demo the CMS systems we're considering for my project. The hard part in all of this is making it sustainable: documenting the setups and configurations that make all of this work, so that other techies can understand, duplicate, and extend the work I've done while I'm here. Fighting tech fires is fun and all, but I won't be here forever, and those tech fires always flare up no matter how robustly the system is designed. I'm thinking an intranet CMS could play that role here.

Can anyone tell me if WDS is really helpful? I'm debating the right way to make sure our facility is adequately covered by WiFi, and right now I've got two AP's that don't really cooperate. (Well, only one answers DHCP, so it's cooperation, but our laptop users still see two different networks.) I don't think I can get our DSL router to do much in the way of WDS, but I can probably get someone to donate another Linksys, then setup WDS between the two, one at the front of the facility and one in back, and just turn off the DSL router's WiFi altogether. Any tips?

Comment from Ben Sheldon on July 11, 2007 - 1:20pm

Hi Sil,

it sounds like you got a good orientation to the organization. I wish we could have our PSO at the lake.

Sounds like you have a lot on your hands and that is always the time for a hardware failure. I hope you've backed up that sole hard drive :-)

I've used WDS at my house and it works pretty well. It does though halve your throughput (since half the bandwidth is reserved for inter-router communication). You probably won't notice this though. I've never used OpenWRT, but I'm a big fan of DD-WRT. Those linksys routers are pretty easy to come by---though watch out since the later hardware revisions won't run the custom linux firmware.

Thanks for the update!

Comment from danielle martin on July 11, 2007 - 4:54pm

Hey Sil,

Great post! I almost feel like I'm sitting there at work with you.

I went through quite a bit of work evaluating custom vs. open-source CMS options earlier this year with the StoriesForChange.net project, so let me know if you need any help in that area. We put out an RFP so I got a good range of proposals from folks doing everything from custom CMS to Plone to Drupal. [We went with somebody doing Drupal, btw, because they included a really great technology planning component to the project planning.]

-Danielle

Comment from Sil Greene on July 11, 2007 - 6:37pm

Well, technically I've installed X-WRT, which is OpenWRT plus the webif^2 configuration interface. DD-WRT is similar, running an OpenWRT kernel with a few added packages and a user-friendly web config. (Using OpenWRT naked is NOT for the faint of heart!)

Comment from Tony Brown on July 12, 2007 - 6:06pm

hey sile u can try to go into the router's config. & set one up as the gateway & one as a router that should fix the problem

Comment from Josh King on July 13, 2007 - 3:39pm

I've never had much opportunity to fiddle with it, but besides WDS you can also install OLSR through OpenWRT's package manager. It's an open-source mesh routing protocol, and while I'm not sure how much set up it requires, it can scale up to clouds of 5-10 nodes pretty easily (don't have exact specs for running it on WRT's, which are a little less than ideal for mesh node hardware). But it might be overkill for just two or three nodes. Of course the easiest way, unless you really need a reason for roaming through the building is just to partition the network. That's what we do here in our building, where we have "OpenUCIMC0-3" in different parts of the building, and they all route to the same physical subnet. If you can't run ethernet lines then you could also run two routers back to back, with one in wireless client mode and the other in AP mode. Finally, you could just get a big antenna. Super Cantennas (http://www.cantenna.com) have pretty great signal penetration, we use two to provide coverage in a pretty huge old stone building. You could hook one up to your first WRT and just blast it through the whole building (depending on layout and the kind of materials your building is made of). Or just use WDS, which would probably work fine too. Just wanted to suggest some other options.

Comment from Sil Greene on July 13, 2007 - 9:47pm

re: partitioning... are all your "OpenUCIMC0-3"'s on separate channels? or are they all sufficiently far away that they can all coexist on the same channel... we're in a building too big for the Linksys by itself, but too small to put to AP's on the same channel. hmmm. the antenna option might actually be the right one; i'll have to dig farther into that.

Comment from Josh King on July 17, 2007 - 4:56pm

I believe they are all on separate channels, though I'd have to go check to be sure. The thing to remember is that channels 1,6,and 11 are the only 802.11b/g channels which are fully orthogonal (if that's the right word). So if you have multiple routers, try and put them on those channels to minimize interference.