Bad Idea #38
(Andrew Armenia's Blog)
Airport WiFi
Today, I'm angry about WiFi. Airport WiFi networks have always annoyed me slightly. I was at Charlotte airport waiting for a connection, having arrived from Chattanooga, where I had configured a proxy server to get around a broken ad-insertion system they were using.
This system was inserting JavaScript into the header of every page I requested. Including things like AJAX requests, which broke a lot of things horribly. So I ssh'd to another box, did an apt-get install squid3, and everything worked well.
Charlotte's WiFi got angry about the fact that I was using a proxy, and started messing with my requests as a result. Do they want to spy on me? Why should they care about whether I'm using a proxy? In any case, they allowed SSH. So 'ssh -L 3148:localhost:3148 my_proxy_server', a quick Firefox reconfig, and I'm back in business.
Why can't we just have WiFi that works, without broken captive portals, ads, or overzealous policies getting in the way? Why can't my Cisco VPN work when I'm in an airport, while I can use SSH to get anywhere I please and tunnel any potentially-evil traffic I want? Even my evil proxy? Probably so some network admin somewhere can feel important rather than feeling helpful or useful. Either that, or no one around here really cares whether or not the WiFi works.
(Via said "evil" proxy, via said SSH tunnel...)
Nasty WinARM ISR bug...
Over the past couple weeks I've been learning the ARM processor, and using the WinARM GCC distribution. I ran across a very insidious bug involving interrupt service routines. Observe the following assembly listing of a function declared with __attribute__((interrupt ("IRQ")))
void _IRQLanding(void) {
101bc: e24ee004 sub lr, lr, #4 ; 0x4
101c0: e92d501f stmdb sp!, {r0, r1, r2, r3, r4, ip, lr}
void (*vic_target)();
vic_target = VICVectAddr;
101c4: e3e04000 mvn r4, #0 ; 0x0
101c8: e5143fcf ldr r3, [r4, #-4047]
vic_target( );
101cc: e1a0e00f mov lr, pc
101d0: e12fff13 bx r3
VICVectAddr = 0x00;
101d4: e3a03000 mov r3, #0 ; 0x0
101d8: e5043fcf str r3, [r4, #-4047]
}
101dc: e8bd501f ldmia sp!, {r0, r1, r2, r3, r4, ip, lr}
101e0: e25ef004 subs pc, lr, #4 ; 0x4
That first sub lr, lr, #4 ends up duplicating the effect of the subs pc, lr, #4 which actually returns from the interrupt. Fortunately, there's an easy enough fix: just add an offset of 4 bytes to your ISR's address when setting up the vectors. This is ugly, but it works...
Oddly enough, this bug seems to have already been reported to the GCC maintainers, and it's been known since 2006. Have they fixed it? No. Why would they do such a thing?
Brake Bleeding on a Mk. V GTI - A Quick How To
If your brakes are getting mushy, your clutch is imprecise, or it's been a couple years, it's time to bleed your brake system!
Important Disclaimer: You're going to be messing with your brake system, so make sure to work carefully. Make sure to go on a test drive away from traffic in case of problems. Brake fluid is caustic, toxic, and flammable. Try not to let it get out. Consult your service manual in addition to these instructions. Work at your own risk, I cannot be held responsible for any damage you cause. The purpose of this howto is simply to document my experience. These instructions may be adaptable to other cars, but make sure to check the service manual.
You'll need the following:
- Some means to pressurize the master cylinder to 15-20 psi. I used a power bleeder made by Motive Products.
- Something in which to catch the brake fluid. I used the catch can that came with the power bleeder, but in a pinch, a soda bottle and some flexible tubing could work.
- Something to suck old brake fluid out of the master cylinder with. The best way I found to accomplish this was with a 1-liter bottle, and about 7 feet of plastic tubing. I drilled a hole in the bottle cap just a bit smaller than the tubing, and pulled the tubing through, creating a seal.
- Some new brake fluid. I used ATE SL.6, which is a low viscosity DOT4 fluid. Get 2 quarts if you've got a manual transmission. One quart should be enough for an automatic.
- A funnel that will reach down to where the master cylinder is. Take a look down there if you don't know what I mean. I cut the bottom off a 2 liter soda bottle and cleaned it out thoroughly.
- 11mm box end wrench, and 9mm as well if you've got a manual.
- Jack, jack stand, tire iron, torque wrench, security adapter, etc...
- Have water on hand in case of any fluid spills. Brake fluid eats paint fast.
Once you've got all of those, you're ready to go. So here's the steps...
- Assemble the power bleeder. Use Teflon tape when connecting the adapter to the main unit, and tighten it well. You do not want any brake fluid escaping here.
- Clean the area around the master cylinder cap. Uncap it, and have a rag ready to go to catch any drips. Leave the filter screen in there - there's no need to take it out, but if you do, it could end badly.
- Use your suction device to suck out as much old brake fluid as possible. If you're using my soda-bottle contraption, start with the cap off the bottle. Squeeze the bottle, then screw on the cap. Put the other end of the hose down at the bottom of the reservoir, and release the pressure on the bottle to create suction. Then, fill it up to the MAX mark with new fluid.
- Hook the power bleeder up to the master cylinder, but leave it empty. Pressurize the system to 15-20 psi and make sure there are no gross leaks. It's normal for the pressure to slowly leak away, but you don't want any geysers of brake fluid coming from the hose connection or the master cylinder cap.
- Open the top of the power bleeder, leaving it connected to the master cylinder. Put in about a quart of fluid if you've got an automatic, about a quart and a half for a manual. Pressurize the system once again.
- Start at the front driver side wheel. Jack up the car (you probably won't need the stand here but it couldn't hurt) and remove the wheel. Locate the bleeder screw on the back side of the caliper. It's got a black rubber cap. Pull off the cap, put your wrench on there, but don't unscrew it yet! Hook up your tubing and catch can first. Then, unscrew it 1/4 turn and let about a cup of fluid (1/4 liter) come out. Retighten the screw securely (if it's open, your brakes will fail.) Put the cap back on. Reinstall the wheel, and lower the car. Repeat at the front passenger side. As you work, make sure to maintain pressure in the system.
- In the rear, the procedure is essentially the same, but make sure to use your jack stand here. The bleeder screws are in a rather awkward spot and require a lot of torque to open. I needed a hammer to get them open, and I was somewhat under the car for this part. Make sure your catch can isn't so full, or your power bleeder so empty, that you don't have time to close the screws.
- If you've got an automatic, when you're on the last wheel, use the opportunity to empty out the hose to the bleeder unit. You can turn the tank sideways, so that air (rather than fluid) flows in. You could also just let it run until it's out, but make sure the master cylinder doesn't run dry.
- If you've got a manual transmission, there's a fifth bleeder screw, located under the hood, for the clutch. This is located on the driver side of the engine, on top of the transmission and down below and left of the battery. Be warned: this one tends to leak, so have rags and water ready. Make sure you've still got pressure in the system for this one (15-20 psi) - I've heard it's important but have yet to hear a good explanation. When you've got the screw open, run around to the driver's seat and manually press the clutch down and pull it back up. Do this about 20 times, leaving the clutch up all the way when you're done. Wait until the air bubbles stop (there may be a lot), then close the screw. You'll also want to turn the power bleeder tank sideways to drain out the bleeder hose during this part.
- Unscrew the cap from the bleeder tank first! Otherwise, brake fluid will probably go everywhere, and that's bad.
- Unhook the bleeder, and cap the master cylinder.
- Shift into neutral, then start the engine. If you've got a manual, being in neutral is important in case you somehow got air into the clutch line. If that were to happen, the clutch could remain engaged even with the pedal pushed all the way to the floor.
- Push the brake pedal down all the way. Go check the bleeder screws for leakage.
- Push the clutch down and make sure that shifting is smooth and there is no grinding of the gears.
- Drive at a slow speed and test brake functionality. Make sure you have a lot of stopping distance during this test in case of problems! Hit the brakes hard to check ABS functionality. In the unlikely event of a problem, don't panic! Judicious application of the handbrake will stop the car. Just make sure to do your initial testing away from situations which could be hazardous if your brakes do fail.
I Can't Drive 55
Today, I was getting on Interstate 787 when I was confronted with a message on a shiny, new digital signage billboard:
"Stop Speeding - Before It Stops You."
I have been convinced for a long time, as I'm sure many others have, that speed limits amount to little more than legalized extortion. A DOT report indicates that speeding is responsible for 1/3 the number of crashes as vehicles stopped in a lane. In fact, the number of crashes caused by "other vehicles traveling in same direction" is approximately equal to the number caused by speeding. This somewhat confirms my long-held suspicions that driving too slow is as dangerous as driving too fast in most conditions.
Upon seeing this billboard (and having spotted a speed trap while traveling the other direction), I proceeded to try an experiment. I got in the right lane and set my speed at about 60mph. As expected, I was immediately passed by approximately everyone on the road. This continued as I accelerated to 70 (Volkswagen-inflated) miles per hour, corresponding to a true speed of about 63mph, or fully 8mph over the speed limit! And accidents on 787 are extremely uncommon. During two summers of commuting the length of the route I only saw one or two severe ones.
I by no means condone speeding in congested or populated areas. According to the report, 36% of accidents occur as a result of vehicle movement at intersections. In fact, I don't condone "speeding" at all - driving anywhere above about 75 (real) mph on 787 today would have been foolish given the traffic. At issue is the definition of "speeding", and its use to harass innocent motorists who are driving safely and effectively. If this report is to be believed, and the police are really concerned about safety, shouldn't they be staking out congested stop signs and traffic lights rather than perfectly good stretches of highway? Shouldn't they be pulling over those who consider it a good idea to drive 50mph in the left lane? If I ever run a red light, even by a split second, I always stop and think "that could have ended very badly in any number of ways." Driving 80mph on a deserted expressway somehow fails to achieve the same effect.
Using git for file synchronization
I've got a few computers, but usually only one of them with me at any given point in time. And what better way to learn about a new technology than to use it in your everyday life? I decided to create a git repository for my personal documents and other files.
The Scenario
I've got 2 laptops: a 17-inch MacBook Pro, and an MSI Wind U123. These both run Mac OS X. My desktop runs Ubuntu 9.04. It made some sense to use the desktop as a central repository, because it's always on. Most of my documents started out on the MBP.
Git Setup
It's easy enough. On the linux box, sudo apt-get install git-core. On the 2 OS X machines, it meant installing Xcode and MacPorts, then sudo port install git-core.
Git Repository Setup
On the desktop,
cd ~ mkdir Documents.git cd Documents.git git init --bare cd Documents git init git remote add origin ~/Documents.git touch dummy # there weren't any files in here yet git add . git commit git push --all origin
On the MBP:
cd ~/Documents git init vim .gitignore: i .DS_Store Microsoft User Data :wq git commit -a git branch mbp git remote add origin desktop:Documents.git git push origin mbp
On the Wind, it's the same but I called the branch wind.
Back to the desktop:
cd ~/Documents git pull git merge origin/wind origin/mbp git push origin master
Now, the process just repeats. To push from the laptops to the desktops, I can just git add .; git commit; git push origin mbp. On the desktop, I can merge the branches from the two laptops back into master. Then, to make sure I've got the latest files on my laptop, I just git pull; git merge origin/master. And since git is distributed, I don't necessarily need to do the merging on the desktops. And in a pinch, I could add some more remotes on the laptops and bypass the desktop altogether. But visions of octopi are beginning to enter my head, so I'll leave it at this for now.
Rails Plugin Testing - A More Complete Formula
I'd written a bit about this previously over at the WTG blog - Rails plugin testing is a horrible, undocumented mess. This is especially true if your plugin uses ActiveRecord, and thus, requires fixtures. To make matters worse, Rails 2.3 changes the way in which fixtures are loaded and used by test cases. I'll be using my authenticates_access plugin as a reference point; the code referenced by this article can be found on the github page.
Getting Started
When you create a plugin using ruby script/generate plugin, a test subdirectory is created. In there, you'll find your_plugin_test.rb and test_helper.rb. your_plugin_test.rb depends on test_helper.rb to set up the correct environment for testing of your plugin, so the first thing it does is to require test_helper.rb. But the default test_helper.rb doesn't do very much, except to load ActiveSupport::TestCase. There are a few more things that need to happen here for ActiveRecord to work right, and to load up some data to run tests on.
Loading More of Rails
The basic test_helper.rb only loads a very small portion of Rails: ActiveSupport::TestCase. We want to make sure that ActiveRecord gets loaded, as well as work around a few bugs in some versions of Rails. So we'll load up some files that Rails forgot about as well as some extras Rails didn't think we needed:
# workaround for rails being on crack (maybe?) require 'test/unit' require 'test/unit/testcase' require 'active_support/testing/setup_and_teardown' # load up ActiveRecord and the plugin in question require 'active_record' require 'active_record/fixtures' require 'authenticates_access' # this is also in the plugin's init.rb... ActiveRecord::Base.class_eval do extend AuthenticatesAccess::ClassMethods end
Database Setup
Next, we'll load up a database.yml file to make ActiveRecord work correctly. Then, we'll proceed to ignore everything in it, and set up a sqlite3 memory database:
ActiveRecord::Base.configurations = YAML::load(File.open(File.join(root_dir, 'database.yml')))
ActiveRecord::Base.establish_connection({
:adapter => 'sqlite3',
:dbfile => ':memory:'
})
Then, set up a database schema:
def build_schema
# define a crude schema
ActiveRecord::Schema.define do
create_table "users", :force => true do |t|
t.column "name", :string
t.column "is_admin", :boolean
t.column "bio", :text
end
create_table "owned_items", :force => true do |t|
t.column "user_id", :integer
t.column "description", :text
end
create_table "admin_items", :force => true do |t|
t.column "description", :text
end
end
end
We'll come back to this subroutine later.
Fixtures
Fixtures represent the data which will be loaded into a database when the tests are run. The fixtures are stored in YAML files, in a directory which you must specify. In Rails 2.3, the code to implement fixtures was moved from ActiveSupport::TestCase into ActiveRecord::TestFixtures, and this module is not included in ActiveSupport::TestCase by default. A workaround is needed to ensure that tests work properly in both versions of Rails. In authenticates_access, my test_helper.rb monkey-patches ActiveSupport::TestCase to do what must be done:
class ActiveSupport::TestCase
# 2.3.2 seems to need this?
begin
include ActiveRecord::TestFixtures
rescue NameError
puts "You appear to be using a pre-2.3 version of Rails. No need to include ActiveRecord::TestFixtures..."
end
self.fixture_path = File.join(File.dirname(__FILE__), "fixtures")
build_schema
self.use_transactional_fixtures = true
self.use_instantiated_fixtures = false
# load up fixtures
fixtures :all
end
First, we try to include ActiveRecord::TestFixtures. If it fails, we just ignore it. Then, we set up the fixture path (in this case, it's a "fixtures" subdirectory in the test directory). Then we call the build_schema routine from before, set a couple of rather undocumented options, and load the fixtures. fixtures :all creates some methods which may be used to access the fixtures. For instance, in the fixtures directory, I've got users.yml like so:
admin: id: 1 name: The Administrator is_admin: true bio: I like writing code and doing other things admin2: id: 2 name: Another Administrator is_admin: true bio: I am also an administrator user1: id: 3 name: User 1 is_admin: false bio: I own some things but not others user2: id: 4 name: User 2 is_admin: false bio: I am like user 1
In authenticates_access_test.rb, these fixtures can be accessed simply by users(:admin) or users(:user1). And that's it for test_helper.rb. Now, on to the fun part.
Actually Testing Your Code
So we've navigated the bureaucracy and set up a framework for tests. Let's get on with them already! It turns out that this part is very easy. Here's an excerpt from authenticates_access_test.rb:
class AuthenticatesAccessTest < ActiveSupport::TestCase
...
test "authenticates_saves :with => :allow_owner on new object" do
ActiveRecord::Base.accessor = users(:user1)
item = OwnedItem.new(:description => "This should work")
assert item.save
assert item.user_id == users(:user1).id
end
test "authenticates_creates negative" do
ActiveRecord::Base.accessor = users(:user1)
user = User.new(
:is_admin => true,
:name => "cracker",
:bio => "I like breaking into things"
)
flunk if user.save
end
...
end
It's as simple as manipulating the fixture objects in some way, and then verifying some assertions. Good tests should cover all of the possible code pathways, but good test design is a subject for another day. When you're done, rake test, then go back and fix your code (and maybe your tests if they're well-and-truly broken) until everything passes.
Update: This is still broken
If you use this plugin in a Rails project, and run rake test:plugins from there, the tests fail! I'm working on a solution to this.
Update 2: It's not as broken as I thought
rake test:plugins fails with this setup because rails runs the plugin tests in the application environment! Why it does this is anyone's guess, but I think it's a bad idea, because plugin tests should test the functionality of the plugin in a controlled environment, and should be self-contained. Attempting to write functional tests for a plugin, to be run in an arbitrary application environment, with an arbitrary model layer, would be exceedingly difficult.
iPhoto Cleansing, and Applescripting using Ruby
I've been storing digital photos on CDs for years, but sometimes the same photos ended up on multiple CDs. This resulted in lots of space being wasted on duplicates when I imported the contents of these CDs into iPhoto.
I've also been meaning to learn how to script applications on the Mac for a while, and it turns out that this can be accomplished using a Ruby gem called RubyOSA. I've used it to develop a solution to my problem, but it's a bit clunky at the moment.
First, go to a terminal and gem install rubyosa - you may also have to downgrade libxml-ruby to 0.5.4. (Use gem list to find out.) If you have a newer version installed, the script will probably die with a bus error. Unfortunately I have no explanation for this yet.
Then, the following script will be of use.
# Find duplicate photos in iPhoto
require 'rubygems'
require 'rbosa'
require 'digest/md5'
iphoto = OSA.app('iPhoto')
md5_table = { }
for photo in iphoto.selection
filename = photo.image_path
puts "processing: #{filename}"
begin
digest = Digest::MD5.new.file(filename).hexdigest
rescue
puts "Error processing #{filename}"
next
end
if md5_table.has_key?(digest)
orig_photo = md5_table[digest][0]
md5_table[digest] << photo
puts "duplicate detected!"
puts "date of original photo: #{orig_photo.date}"
puts "date of this photo: #{photo.date}"
photo.comment = "DUPDUPDUPDUPDUP"
else
md5_table[digest] = [ photo ]
photo.comment = ""
end
end
dupes = 0
md5_table.keys.each do |key|
photos = md5_table[key]
if photos.length > 1
dupes += 1
end
end
puts "in total, found #{dupes} photos with multiple copies"
Save this as a text file (e.g. iphoto_dupes.rb), select the offending photos in iPhoto (or just select all), and run the script from the terminal with ruby iphoto_dupes.rb. The code will put "DUPDUPDUPDUPDUP" in the comment field of any photos that are exactly identical to one that had already been found. From there, you can search for "DUPDUPDUPDUPDUP" in iPhoto and delete the duplicates. (If you use iPhoto comments, you'll need to modify the script slightly to keep from overwriting them.)
I'll be looking into ways to package this script and the required Ruby gems up in a more convenient form. But meanwhile, hope this helps...
OK, so that didn't take long
I just realized that I'm angry. Angry about Ruby's sometimes-arcane, sometimes-nonexistent error messages.
Earlier today, I was working on some code for my research project. The purpose of this code was to collect some data from a UDP stream and route it to one or more destinations (implemented as Ruby classes). To accomplish this, I was using threads. Easy enough, right? my_thread = Thread.new { myfunc }. I cranked out a working prototype in a couple hours. Isn't Ruby great?
I realized that it'd be a good idea to refactor a bit and move the destination classes into their own separate files. In so doing, I managed to forget a require or to. No problem, Ruby will just throw an exception. Right?
Ruby threw a few exceptions. But far more interesting was the one it didn't throw. In my main program I had the UDP receiver thread. And in my refactoring I'd moved a require out of the main program. As it turns out, the module I'd neglected to require was only used in the UDP receiver thread, which runs something like this...
Thread.new do
while true
msg, src=sock.recvfrom(1024)
use_the_missing_module_to_parse(msg)
end
end
So as soon as data would be received, the thread would of course throw an exception. Or would it?
I was receiving a very cryptic error message indicating that a deadlock had been detected.
/usr/local/lib/ruby/1.9.1/thread.rb:184:in `sleep': deadlock detected (fatal) from /usr/local/lib/ruby/1.9.1/thread.rb:184:in `block in pop' from :8:in `synchronize' from /usr/local/lib/ruby/1.9.1/thread.rb:180:in `pop' from spsp_router.rb:51:in `'
Then I got very angry at Ruby for telling me there was a deadlock in code whose fundamental logic hadn't changed. While deadlocks may indeed be tricky to debug, a deadlock was not the direct cause of the problem. Indeed, this made the entire situation more difficult to debug, since Ruby managed to swallow the original exception which directly caused the failure. So I leave with one question: What were they thinking?
Filler Material
OK, so the shiny new blog controller now works. Hooray! I've made something working in Rails. Now for the tough part: finding something to write about.
I've come to the conclusion that too many people use blogs to write about trivial subjects (case in point: so many abused Twitter accounts), so I've decided that every post on this blog should be about something worthwhile. This means that posts probably won't be all that frequent. But expect them to be on a great variety of subjects, from electronics to Ruby on Rails to student life at RPI to the world at large.
I find it unfortunate that I can't stay true to my word with the first post. My mind has become too addled with thoughts of models, views, and controllers; indeed, a certain annoying tune about those things has been playing in my head, drowning out all higher-level constructs like words, sentences, or ideas. So enjoy the blank space until next time.