Moonbase

For you.

BAMAN PIDERMAN

Interview with Dali

(Both these gems are by Alexander Butera; I hope we get to see more from him in the future!)

I don’t have a whole lot to say about science blogger PZ Myers’ public desecration of an allegedly consecrated host. I think it is fair to take him at his word that he does not seriously entertain the idea that a piece of bread could by divine fiat become the Body of Christ (which brings to mind Christ’s response the first time people started sticking nails in it ). In any case, that much is between him and God, and I generally wouldn’t expect a non-Catholic to have a Catholic perspective on the issue.

What I do hope most people would recognize is the basic immaturity of the act, even from a strictly secular point of view: in such terms, it’s akin to a grade-schooler taking from another child the special sandwich the child’s mother had prepared for him and grinding it into the dust because it was “stupid”. Of an adult academic, such an act would suggest an exceptionally poor standard of public behavior. That Myers’ act was carefully premeditated, and that the intended recipients of the Eucharist place an even higher value on it, that only makes that issue worse.

I realize that Myers had taken it upon himself to act in retaliation for threats made by a few of my co-religionists in an unrelated Florida incident. But if certain people are acting like vindictive children, an adult response does not mean similarly immature behavior, and an adult response most certainly does not mean taking it out on an entire religious group.

Update: Based on the replies I am getting, it seems that there are four issues which I need to clear up.

First, Myers (at least if his own claims are to be believed) didn’t simply obtain a wafer of the same sort of bread which is used in Catholic rites (which, absent any confusion, nobody would really care about), he specifically sought out a wafer that had actually been in the posession of the Church and had been consecrated by a Catholic priest.

Second, there isn’t a legitimate way to remove a consecrated host from the Church’s custody. The Church does not relinquish custody of a host distributed at Mass until it has actually been eaten. To this end, Catholic churches train their staff—ushers, priests, deacons, and extraordinary ministers—to observe and ensure that the consecrated hosts distributed are consumed on the spot as intended (even if, as this incident shows, they are not always adequately vigilant).

Third, even if the removal of the host in this case might have constituted petty theft (or grand theft, if the “buy it now” price on that one eBay auction is indicative of the monetary value of consecrated hosts), I’m deliberately avoiding legal considerations. From a simple, human, secular standpoint, the basic issue is that it’s inappropriate for a mature adult to take an article of extreme sentimental value to its intended recipients and destroy it in public as a deliberate act of spite.

Fourth, the original threats Myers was retaliating against were directed at a kid in Florida, not him personally. Professor Myers read about this in the media like the rest of us, and decided to take it upon himself to strike a blow for … something. (As far as I can tell, his two main accomplishments so far have been to provoke some of the same morons to start sending him threats, and to offend an awful lot of uninvolved Catholics.)

I think this article can speak for itself:

The military trainers who came to Guantánamo Bay in December 2002 based an entire interrogation class on a chart showing the effects of “coercive management techniques” for possible use on prisoners, including “sleep deprivation,” “prolonged constraint,” and “exposure.”

What the trainers did not say, and may not have known, was that their chart had been copied verbatim from a 1957 Air Force study of Chinese Communist techniques used during the Korean War to obtain confessions, many of them false, from American prisoners.

...

“What makes this document doubly stunning is that these were techniques to get false confessions,” Levin said. “People say we need intelligence, and we do. But we don’t need false intelligence.”

...

The only change made in the chart presented at Guantánamo was to drop its original title: “Communist Coercive Methods for Eliciting Individual Compliance.”

Apple’s announcement of their Grand Central technology sounds interesting, but it’s pretty light on details. Here’s my wild guess, based on the intersection of what they’ve hinted at and what I would do if I were in their shoes: my guess is that they’re integrating support for fine-grained scheduling (as in e.g. TBB) into the OS scheduler, which would enable smarter global decisions about things like work balancing and cache warmth.

Think M:N threads with better kernel support, but minus the usual attempt to make the “userspace” portion of the scheduler preemptive. If they’re smart, they’ll also include non-blocking/asynchronous equivalents of standard blocking APIs, though in principle with sufficient kernel support they could also permit tasks to use some blocking operations in more transparent ways.

I guess we’ll see how good my guess was in a year or so.

The Museum of RetroTechnology is a compilation of various “steampunk-type” technologies which were or are in use in the real world.

(Hat Tip: Nathan)

(On YouTube)

The Rebellion Within is a long but really fascinating and somewhat surprising look at the origins and recent development of Jihadist organizations since the 1970s.

(Hat Tip: Tim Bray)

There was a lot of buzz about WiiWare on the message boards when it first came out, as if it were something with a low barrier to entry comparable to, say, Microsoft’s XNA.

It’s not, okay guys? WiiWare is just an alternate distribution channel for Wii games, complimenting pressed discs in stores. Since WiiWare games are necessarily smaller in scope and the distribution mechanism doesn’t involve the same financial commitments as discs, Nintendo will certainly approve titles for WiiWare that wouldn’t be worth their while to approve as a UPC to go on shelves, and perhaps will also be willing to take a little more risk with developers who have less proven experience, but that’s about the extent of the difference to regular Wii development.

That said, if you’re an indie game development company leasing commercial office space, with some published titles under your belt (or at least a wildly popular flash games site), and for some inexplicable reason you managed to get that far without knowing where to look for these things, the Nintendo SDSG has a list of requirements and an application form posted on their website.

I hope that helps clear up any confusion.

Harley Davidson supposedly puts a lot of care into the sound of their motorcycle exhaust systems. What if a Harley exhaust system or three were rigged to a pipe organ to supply the nice low notes you can’t normally get?

My brilliant friend Nathan took some time to simulate the result using the Aeolus organ simulator to record a performance of BVW540 (a Bach Toccata and Fugue in F Major) using the “Harley stop”.

After Andrea O. K. Wright’s wildly popular (popular as in she had to schedule a second session after the first one literally overflowed with people) RailsConf talk it seems like now is a good time to give an update on my plans for the Omnibus Concurrency Library, especially after I’ve neglected it for so long.

At this point, the next version of the library is likely to be a complete rewrite, licensed under an MIT-style license similar to Rubinius’. It will aim to support at least the following Ruby implementations, with others following as time and interest permit:

• Ruby 1.8
• Ruby 1.9
• Rubinus
• JRuby

Portability

Presently, only JRuby presents opportunities for improved parallel performance with Omnibus’ thread-centric implementation; under Ruby 1.8, 1.9, and Rubinius, performance will necessarily be equal to or less than single-threaded performance. Why support them at all? There are two main reasons:

The first reason to support green-threaded Ruby implementations is for the sake of portability, allowing the same program that runs under JRuby on a multi-core behemoth from Sun to scale all the way down to a single-CPU machine running Ruby 1.8 with its green threads.

Secondly, concurrent programming abstractions are often convenient ways to structure programs even on single-processor systems. (If they weren’t, nobody would ever bother with Ruby 1.8’s green threads.)

Additionally, the concurrency picture for some of these Ruby implementations is not necessarily bleak in the long-term: Rubinius already permits taking advantage of multiple threads per process in separate VMs. A future version of Rubinius may offer more fine-grained multicore support, and a future version of Omnibus may find ways to exploit things like the existing per-VM multicore support, or even include support for fork-based parallelism for some purposes.

Organization

Rather than Omnibus remaining a monolithic library, I’m also going to be breaking things up in order to enable individual components to be released as separate gems (and hopefully in a more timely fashion):

• concurrent – “omnibus” gem which pulls in the others as dependencies
• concurrent-actors – Actors for Ruby
• concurrent-futures – Futures for Ruby
• concurrent-joins – Join calculus for Ruby
• concurrent-parallel – Parallel algorithms and parallel extensions to core/stdlib classes (corresponding roughly to TBB parallel algorithms)
• concurrent-primitives – Grab-bag of simple primitives like channels
• concurrent-selectable – Semaphore and a few other primitives which can be waited on using existing event-based IO libraries, or just IO.select (DONE)
• concurrent-sequential – Force strict sequential execution of asynchronous tasks, even in the face of multiple threads or reentrancy. Don’t laugh, it’s useful.
• concurrent-tasks – Task-oriented parallelism (similar to TBB tasks)

Major Changes and Incompatibilities

One of the major changes coming is that the library will handle asynchronous tasks (e.g. from futures, or joins) by dispatching them to a TBB-like task scheduler instead of simply spawning a new thread for each task. In most cases this means that the creation of continuation tasks is the preferred alternative to blocking, although there will still be provisions to support blocking tasks. The largest effect of this change will be the elimination of transparent futures from the core library, since they have the effect of causing arbitrary tasks to block, and have consistently been the most difficult feature of the library to implement portably anyway.

There will also be some less drastic, but incompatible, changes to the rest of the existing APIs. In particular, parallel algorithms will take grain sizes rather than a count of threads to spawn, and the actor API will be modified to match the current Rubinius API more closely.

Timeline

Especially since Omnibus is a spare time project for me at the moment, I can’t commit to a timetable for the new release, but at the moment I’m aiming to have the new release out in 2-3 months. Watch this space.

(Incidentally, feel free to contact me if you’re interested in funding some aspect of Omnibus development. I am committed to finishing Omnibus regardless, but obviously paid projects have to take precedence.)

I’ve been meaning to blog this for a while…

Tony Arcieri and I devised an object protocol that any object acting like an actor in Ruby ought to implement in order to interoperate with our actor implementations. It is implemented, more or less, in both Revactor and the Rubinius actor implementation, and I plan on refitting Omnibus actors to satisfy it as well.

(Interoperate, here, means simply to be capable of receiving messages from any code which observes this protocol, and also having the ability to participate in linking and supervision tree relationships with other objects implementing the protocol. Protocol is meant in the Smalltalk sense of the term, which is essentially a “duck type”.)

Anyway, here is some pseudo-Ruby sketching out the protocol in all its glory. It consists of four methods:

protocol Actor
  # Submits the given message to this actor's mailbox.
  #
  # Messages submitted to a dead actor are silently ignored.
  #
  def <<(message) ; end

  # Notifies this actor that other_actor has linked with
  # it.  It represents "half" of the linking process,
  # merely instructing this actor to add other_actor to
  # its set of linked actors if it has not already been
  # added.  In order to link two actors (we'll call them
  # a and b), an actor implementation should perform:
  #
  #  a.notify_link(b)
  #  b.notify_link(a)
  #
  # If an actor receiving a link notification is dead,
  # when its implementation permits it should respond
  # to other_actor's link attempt by calling:
  #
  #   other_actor.notify_exited(self, reason)
  #
  # (See the description of notify_exited for
  # discussion of exit reasons.)
  #
  # Aside from dead actors responding with exit
  # notifications every time, notify_link should be
  # idempotent:  repeated attempts to link the same
  # actor should have no additional effect.  An actor is
  # either linked or it is not; there is no notion of
  # "link count".
  #
  def notify_link(other_actor) ; end

  # Notifies this actor that other_actor has unlinked
  # from it.  It represents "half" of the unlinking
  # process, and merely instructs this actor to remove
  # other_actor from its set of linked actors if the
  # set of linked actors currently contains it.
  #
  # Similarly to linking, an actor implementation
  # should perform unlinking as follows:
  #
  #  a.notify_unlink(b)
  #  b.notify_unlink(a)
  #
  # Unlike link notifications, an unlink notification
  # sent to a dead actor has no extra effect.
  # 
  def notify_unlink(other_actor) ; end

  # Notifies this actor that other_actor has died, either
  # by exiting normally or as the result of an error.
  # If the dead actor exited normally, reason should be
  # nil, otherwise it should be an object which
  # stringifies to a description of the error (often an
  # Exception object, but even a simple string is
  # allowed).
  #
  # When an actor dies, it should call notify_exited on
  # all of the actors linked to it *at its time of death*
  # in order to notify them that it has exited.
  #
  # How an actor responds to the reciept of an exit
  # notification is outside the scope of this protocol,
  # although it is suggested (not required) that the
  # default behavior be for the receiving actor to
  # terminate itself, as that is most useful behavior
  # for implementing supervision trees.  However, it
  # is permissible for implementations to ignore exit
  # notifications.
  #
  # Exit notifications sent to an actor that is itself
  # already dead should always be silently ignored.
  # 
  def notify_exited(other_actor, reason) ; end
end

Beyond that, the protocol places no requirements on the scheduling or concurrency properties of the actor represented by an object implementing this protocol—it need not even be an actor in the usual sense.

atomic_ops is a library of portable primitives for building lockfree multithreaded programs in C, by Hans Boehm of libgc fame. The basic atomic operations portion of the library is MIT-licensed, although it also includes a GPLed addon library which provides nearly lockfree stacks and a lockfree malloc implementation.

Update: Hans Boehm wrote to say that the older distributions are to be avoided in favor of the copy of the library in the libgc tree, which he has been keeping more up-to-date. It is available from libgc CVS at:

:pserver:anonymous@bdwgc.cvs.sourceforge.net:/cvsroot/bdwgc/libatomic_ops-1.2

libatomic_ops also appears to be available in many Debian-based distributions as libatomic-ops-dev, in a version based on the last official release but with a lot of additional patches.

(This is a follow-on to my post The Adobe Flash Player Deadlock)

I’ve been very pleased to see that, after my original post on March 9, many of my criticisms have been addressed, and I’ve learned that some were simply founded on poor communication. Let’s run through some of them.

Criticism: There is no way for the general public to report flash player bugs.

Originally, your options for reporting bugs were posting comments on a certain blog, or filling out the Adobe feature request form at http://www.adobe.com/go/wish. The blog wasn’t particularly encouraging since few of the issues raised there got addressed or even received a positive response (I’m also not linking to the blog because towards the end the exchanges between frustrated Linux users and a beleaguered Adobe project manager make both sides look needlessly bad).

I had also not been aware of the wishlist form as an accepted avenue for reporting bugs, but the current version of the page makes that a little more clear and easier to find in a search on Adobe’s site. However, the wishlist form still isn’t very ideal. Rather than a black hole to send “wishes” into, I’d really been hoping for a public bug tracker where you can actually see the fate of your problem report. Even Sun, in their most obnoxious days, had such a thing for Java.

Thankfully, as of April 8, Adobe finally has a public bug tracker for flash: http://bugs.adobe.com/flashplayer.

This criticism is now completely addressed.

(Many thanks to Marnen for filling me in on this point.)

Criticism: Adobe is not devoting sufficient resources to the Linux Flash player for it to remain viable.

I had based this criticism on Adobe’s public statements and history so far. After an initially promising release of Flash 9 (Linux Flash had previously languished buggily at verison 7 for an extended time), subsequent Flash 9 releases were increasingly infrequent and full of new regressions. Adobe was supposedly going to make a big push for Linux with the release of AIR, but when Adobe actually released AIR 1.0 in February, they omitted support for the Linux platform since (according to the AIR FAQ at the time) they had to “wait on the core Flash Player’s support for Linux to be finalized.” All these things together painted a very ominious picture for the future of Flash on Linux.

On March 30, however, Adobe released an alpha version of AIR for Linux, showing that they were making a serious effort to support the platform. I suspect that the reason for the falloff in Flash 9 support is the result of Adobe reallocating programming resources to Flash 10/AIR, although it would be nice if Adobe publicly announced their plans the same way that they did for 64-bit Photoshop, so people wouldn’t be left to conclude that, absent released software, they didn’t have any serious plans at all.

This criticism is probably unfounded to the extent that Adobe is doing the best they reasonably can as long as they are not accepting outside help by opening development of their own player (which itself would require a significant initial investment of resources that Adobe may not have).

However…

Criticism: There is no finalized release of the Flash player for Linux which is stable enough to use for development work.

It’s still the case that the latest Flash Player 9 takes out my browser multiple times an hour if I use it heavily (FlashBlock has helped mitigate this issue for casual use). Since Linux is my preferred development platform, there’s no way I could see myself developing for it until this changes.

(A few people have asked me why I think Adobe has to be altrustic and expend resources supporting Linux. They don’t, of course. But then I don’t have to be altrustic and bend over backwards to develop for an RIA platform that doesn’t have good support for the development platform I normally use either. Remember that the original post was about why I personally didn’t want to develop for Adobe’s platform.)

For now this criticism remains unaddressed, although with the alpha release of AIR for Linux I have hope that it might be addressed this year.

Criticism: Adobe is putting up roadblocks for open source Flash player implementations.

Even when Adobe finally published specifications for the flash format, they carried restrictions which prevented using them to implement your own flash player. Interviews with Adobe employees in the past had also indicated that Adobe was concerned about retaining control of its platform. Worse, in March they had announced their intent to make DRM part of the flash platform, introducing an additional obstacle for Open Source players.

However, just this month (May), Adobe launched the Open Screen Project and removed the restrictions on the use of their specifications. Some have called this a PR stunt, since the way it worked out Adobe hadn’t relaxed the restrictions on the specifications until all the information published in them had been reverse-engineered by the developer community anyway. However, I think calling it a PR stunt misses an important point—the Open Screen Project is specifically a signal that Adobe is no longer directly hostile to alternate Flash Player implementations.

This criticism is largely addressed; DRM remains an issue on the horizion, but Adobe’s public stance against non-Adobe players has clearly changed.

Criticism: Adobe doesn’t consider broad platform support for Flash to be important.

Historically, aside from Flash Lite (which is fairly different to the regular Flash platform), Adobe’s official support for Flash has mostly extended to PPC OS X and x86-32 OS X and Windows. There’s been the rather flaky and inconsistent support for x86-32 Linux (and the zombie-like revival of some version of the PPC Flash 7 plugin for the Wii), but that’s about it. The thing is, until you are able to support a certain number of platforms, the incremental cost of supporting each additional platform is extremely high. There’s essentially a knee in the graph, and Adobe is still quite far from reaching it.

As Sun discovered with Java, truly broad platform support is really only possible with an open source implementation of the platform. The community-developed zero-assembler Hotspot has enabled the JVM to be brought to a wide variety of new platforms which didn’t have Sun Java before with extremely little initial effort, and soon may start leveraging LLVM for cross-platform JIT performance. Flash will not be able to catch up in terms of platform support until at least one solid open source implementation of the Flash platform is available. On the evidence of the Open Screen Project, and their previous open-sourcing of the Tamarin runtime, I think Adobe finally realizes this.

I think this criticism was unfounded. Their release of Tamarin in November ought to have been a clue to me that they were waking up about this.

Job stress had really gotten to me the past couple months, to the point where I really didn’t have the time or energy to blog. Now that I’ve made some needed changes, I’ve got quite a lot of ideas piled up to blog about. I will try to pace myself and do it in small chunks, to avoid a yeggesplosion.

I’d had grandiose plans for a series of blog posts over the Triduum this year, but I never finished Good Friday’s post in time and it was downhill from there. Eh well. However, while I’m neither Anglican, Episcopal, nor a great fan of Rowan Williams, I recently read the sermon he gave this past Easter Sunday and thought it was unusually good. It’s no longer Easter, but it’s still the Easter season, so I may as well share:

The Archbishop of Canturbury’s Easter Day Sermon

This is the awesomest ad:

Monster – Legs

(Update: see Adobe Does Not Suck for an update on these criticisms.)

I don’t use proprietary software very often; one of the few pieces of proprietary software that I do use with some frequency is the proprietary Adobe Flash Player plugin. On a daily basis, it reminds me why I try to avoid proprietary software in the first place.

One illustrative point: I’m sure most people who have used the plugin on Linux experienced occasional browser lockups when exiting a page which used Flash. I experience it frequently, and everyone I’ve asked has had the same issue. It’s been a problem with the 9.0 releases of the plugin for a long time, but Adobe’s never done anything about it. Either they don’t know, or they don’t care.

I can’t rule out not knowing. I’d love to report the bug, but Adobe doesn’t provide an obvious way to do that without going through their “complimentary product support” channels—which require a registered Adobe product. All I want to do is put in a PR! Most open source projects are more receptive to bug reports (and that isn’t saying much).

Of course, if I did put in a PR I couldn’t really be more specific than “the flash player sometimes deadlocks when tearing down a plugin instance, locking up the entire browser.” I don’t have the source available to me to examine the issue in any more detail, and the EULA I agreed to when installing the plugin is rather stringent about reverse-engineering, so it’s not like I could legitimately go in with gdb to see what the deal is either.

This is hardly the only annoyance, either. Non-IA32 platforms have been left in the dust for a long time, and in recent revisions of the 9.0 plugin for Linux, there’s at least an additional livelock, worsening memory leaks, and severe performance regressions in the video pipeline. (Incidentally, if you’re having memory problems with Firefox, try removing the Flash plugin and see the kind of difference it makes.)

If the Flash plugin were open source and the worst issues had remained unresolved for this long, I’d try to resolve some of them myself. I’ve got a decent amount of experience with concurrent programming, and more likely than not the issues are fairly straightforward. This is not because I feel any special obligation to Adobe to do their work for them. I’d be doing this for my own sake, and for the sake of the other people who have to deal with this stuff all the time.

Adobe’s decision to open source their Tamarin JavaScript runtime had me hopeful that they would eventually also open source the Flash runtime (at least just the plugin), but their more recent decision to incorporate DRM into Flash is discouraging. They could still open source the non-DRM portions, but most likely my only hope for an open source Flash plugin is something like Gnash—which I can’t safely contribute to now that I’ve agreed to Adobe’s EULA for the proprietary plugin. In the long term the DRM decision is also likely to cut open source players off at the knees since their ability to play the most popular Flash content will be strictly limited. If Adobe isn’t going to give us an open platform to play Flash content, nobody else can either—is that it? Beacuse that’s the appearance it gives. You get it from Adobe, or not at all—and there’s an awful lot that Adobe simply isn’t providing.

Thanks Adobe; you’ve managed to alienate another developer. I’m going to stick to authoring software and content for platforms whose vendors don’t play these kinds of stupid control games, so I don’t lock my own users into your twisted little world. Sun’s entirely open platform is looking much better these days, and they’ve finally got people working to address the gross deficiencies of Java applets that gave Flash its opening in the first place. I’ve done Flash development in the past—the reason Flash dominates right now is not because it’s great platform to use and develop for, but rather only because the alternatives have been so much worse. That circumstance is, thankfully, beginning to change.

Garritan has put together an online version of Rimsky-Korsakov’s classic book on orchestration. With commentary on the original text, and the ability to play back score fragments and follow along in your browser if you have Flash, it’s a really awesome resource.

PantoGraph is a prototype 3d rendering engine that outputs SVG and is implemented as a Blender Python script.

(Hat Tip: BlenderNation)

My PGP keys expired recently; the new public key is available from the usual place. The fingerprint of the new key should be E829 4985 45A1 5870 7239 A6FF 1293 0555 EF78 AF6E.

Those with whom I’ve been maintaining key trust relationships, see me offline about signing.

On YouTube

SoundManager 2 is a rather nice BSD-licensed JavaScript library which employs a small Flash shim to provide a straightforward JavaScript API for sound playback.

Use responsibly, kids.

Intel’s go parallel subsite on DevX has a rather nice article up about designing threaded applications. Highly recommended.

I’d also suggest browsing around the go parallel section itself; it’s not bad reading even if it has an (understandable) bias towards Intel tools and technologies.

(Hat Tip: Jeff Turcotte)

There’s a new interview with me up on InfoQ about Ruby concurrency, Rubinius, and the Actor Model.

Happy New Year! Today is the first day of the seventh year of the second millennium, and the Solemnity of Mary, Mother of God. Pace!

Zed Shaw writes about what makes the lua project work. I think he’s dead on, because it matches my own experiences very well. I think one of the reasons that Inkscape is as succesful as it is is that we work very hard to foster a “collaborator-style” community and integrate newcomers into that culture.

A Google Tech Talk by Hans Boehm.

(Thanks Bob!)

Merry Christmas!

Yes, it’s exactly what it says on the tin.

Introduction

Discussions of multithreaded programming and the Java Memory Model often get bogged down in discussion of complex compiler optimizations. I don’t think it needs to get that complicated—if all you want to do is write correct multithreaded programs, there’s a pretty simple mental model you can follow.

How to Think about Memory

Each thread has its own view of memory such that two threads can disagree on the contents of the same field. Writes made to a field by one thread may bleed through and become visible to others, but they may also appear only transiently, in the wrong order (even before they are “supposed” to happen!), or not at all. This is permitted for performance reasons.

However, Java does provide a few specific ways for threads to reliably communicate writes to one another…

The Tools at Your Disposal

There are five ways to explicitly ensure that one thread’s writes up to a certain point will be visible (and in the right order!) to other threads. Each one uses a particular sort of language primitive as a rendezvous for a writing thread to “publish” its writes, and for other threads to “receive” them:

Note that for the rendezvous to happen, it must be the same thread, the same field, or the same monitor on both the “publishing” and “receiving” sides.

Really, at all levels of abstraction, all correct concurrent programs share a particular characteristic: threads do not communicate with each other at all except when they mutually agree to communicate at common rendezvous points. Any truly unilateral attempt at communication (like Thread.suspend) is inherently unsafe.

Applying Your Knowledge

Let’s say we’ve got a class like this:

class Counter {
    private int count = 0;
    public synchronized void increment() {
        count++;
    }
    public synchronized int getCount() {
        return count;
    }
}

Making increment synchronized ensures not only that the increment of count (which involves a read and then a write of the modified value) isn’t disturbed, it also ensures that the modified value of count is “published” where other threads can potentially see it.

Likewise, making getCount synchronized ensures that the thread “receives” any “published” changes to the value of count when the method body is entered.

Both synchronized declarations are necessary for this code to work as expected. Of course, since getCount is simply reading a single int field (which is already an atomic operation), there is another way to approach this:

class Counter {
    private volatile int count = 0;
    public synchronized void increment() {
        count++;
    }
    public int getCount() {
        return count;
    }
}

This means that increment will “publish” twice: once when it writes to count, and once when it exits the synchronized method. This has no ill effects and only a very minor peformance penalty.

getCount only “receives” once: when it reads count. Since it is not synchronized by a monitor, concurrent calls for getCount do not have to wait for each other to complete. This can result in improved performance for readers.

Note that, had increment performed any writes to other fields after the write to count, those writes would not be safely visible to callers of getCount; the second “publication” which happens upon existing the synchronized method is via the object’s monitor, not the volatile field.

The Consequences

If you do not properly coordinate the “publication” and “reception” of writes between threads that share fields, then your program will fail transiently. It is more likely to do so under production load on high-end hardware than it is during testing on your development machine.

You don’t want that. Neither does your boss.

Further Reading

The JSR 133 FAQ goes into more detail about the memory model itself. Like the Memory Model document, it goes a little more into the effects of specific kinds of optimizations.

One Last Thing

If you ever do get dragged into a discussion of compiler and runtime optimizations, it’s important to remember several things:

1. Disassembling .class files doesn’t tell you much about what will happen at runtime: javac performs very few optimizations, leaving most of them up to the interpreter and JIT compiler.
2. While the interpreter and the JIT compiler are prohibited from performing certain optimizations like reordering dependent reads or writes, the CPU itself is not.
3. A few CPU architectures can and will reorder dependent memory operations and get away with it—unless you are writing improperly synchronized programs.
4. Still, today’s hardware is often fairly forgiving about multithreading. As we’ve run out of room to scale “up” and have to scale “out” instead, future hardware will have to sacrifice that forgiveness for increased concurrent performance. Unless you want to rewrite all your code in five years, do it right the first time.

One Other Thing

Try using the classes in java.util.concurrent and java.util.concurrent.atomic before rolling your own custom solutions using synchronzied and volatile. The java.util.concurrent classes are likely to be both correct and as fast as possible while still maintaining correctness (which is seldom easy to achieve). They’re built on top of the basic primitives themselves, so they have similar characteristics: for example, writing to a java.util.concurrent queue “publishes” to the queue’s readers.

As the use of Swing with JRuby is becoming increasingly popular, are there any good Swing resources out there directed at Ruby programmers? If not, I may have to take a stab at writing some material…

So, I was thinking today—what makes a programming language a scripting language? There are three characteristics which seem to be typical:

1. Most normal use of the language does not involve an explicit (user-visible) compilation phase
2. Expressions or statements are valid top-level productions in the language’s grammar
3. The language has a very rich (and possibly somewhat domain-specific) standard library

Any good counterexamples? That is, are there any languages commonly considered scripting languages which do not meet all three criteria? Conversely, are there languages which are not commonly considered scripting languages which do meet all of them?

Here’s a brief overview of how I set up and use git-svn to commit to JRuby.

Initial Setup

If you want the whole history of JRuby, branches and all, just do:

git-svn clone -Ttrunk/jruby -ttags -bbranches --username mental https://svn.codehaus.org/jruby jruby

...which will create the directory jruby and clone the entire repository history into it, then check out a working copy from trunk. I typically just work with individual branches, though. My own initial setup was just to work with trunk:

git-svn clone -Ttrunk/jruby --username mental https://svn.codehaus.org/jruby jruby

You don’t actually need the whole history, but it is occasionally incredibly convenient to have, and once you clone you’re mostly stuck.

My username is actually the same locally as it is remotely, so I didn’t need to use the --username option; I just included it for completeness.

Day-to-day Operation

Before starting, I make sure I’m on the right git branch and have no uncommitted changes. Then I can make sure that I have current version of the mirrored Subversion branch I want to work against (here, remotes/trunk):

git-svn fetch -i trunk
git-rebase remotes/trunk

Then I can just git-commit to my heart’s content. When I’m ready to push my changes back to Subversion, I can just run:

git-svn dcommit -i trunk

If there are any conflicts with new changes in Subversion, it will fail and you will need to fetch and rebase again (resolving the conflicts manually —see the git documentation) before you will be able to commit to Subversion. I usually do this proactively, to make sure that I have a chance to test the commits in their rebased form.

Topic Branches

Sometimes I have a number of things I’m working on at once. In those cases I’ll create a topic branch to work on a feature separately, rather than working in a regular persistent branch like master:

git-checkout -b topic/thingy-feature remotes/trunk

At this point topic/thingy-feature becomes the current branch and the procedure works the same as above. Once I’m done working on the feature I usually switch back to one of the “regular” branches and delete the topic branch with:

git-checkout master
git-branch -d topic/thingy-feature

Combining Commits

If I’ve been working in git for a while without commiting to Subversion, I may have a non-trivial sequence of commits saved up (I tend to commit in small increments as I work), where only the final one really reflects a working state. To collapse them into a single commit before dcommitting, I can fetch/rebase and then do:

git-reset --soft remotes/trunk
git-commit -c ORIG_HEAD

I usually don’t need to do this as I try to integrate frequently with trunk, though.

Merges and Non-Linear History

Linear histories are trivial, but merges and Subversion don’t really get along. This extends to git-svn as well. That’s material for another post, however.

I’ve been listening to pianist-composer Sean Mahnken’s album Patterns for the past couple days and I’m enjoying it quite a bit. You can hear some of the songs on his web site.

Erez Petrank has quite a lot of interesting garbage collection projects and papers up on his projects page.

Now that I’ve discoverd StumbleVideo, I guess I’m not getting much practical done tonight. While nowhere near as good as One Rat Short, here are a few other selections I liked:

• Moutons – fluffy sheep! underwater!
• I Lived on the Moon – a girl versus the angry sun gnomes
• Over Time – puppets cope with loss
• The Pier – a fisherman gets more than he bargains for (n.b. slightly gory)
• 9 – the last of his kind, number nine prepares to face down the creature that killed the others

(Over Time is probably my favorite.)

I discovered One Rat Short via Stumbleupon Video today; it has to be one of the best-made animated shorts I’ve seen in a very long time.

I really hope we get to see more from Charlex Films in the future.

Thanks to quite a lot of readers writing in, I now have a much better picture of what’s going on with charging my new phone. There are really two considerations when charging a device via USB:

First, not all USB ports are prepared to power devices (like charging phones) with high power requirements. There are various levels of peak power consumption defined by the USB specification, so it’s necessary to match the device to an appropriate USB host which can meet its specific power requirements.

Secondly (and this is evidently my real problem), many Motorola devices specifically check whether the USB port belongs to a Motorola-manufactured device; this is signaled by shorting certain pins, as documented here. Phones may also ignore this hardware check and charge from a regular USB port if the host computer has special software installed which responds with the Motorola “secret handshake”; a Linux machine running moto4lin appears to be sufficient for this purpose.

I had to urgently replace my phone recently, and decided to get a Motorola V195s since it had no camera (a requirement imposed by my job) and in my particular situation was only about $9 after rebate. It’s not a great phone, but I needed a phone quickly which could reliably make calls.

One thing I was pleased to find, however, was that the phone charged via its mini-USB port. Yay! No more worrying about losing the wall dongle.

Or so I thought. I spent the entire day trying to get it to charge from various USB charging dongles and USB ports on different hubs and computers. I even checked the voltage/current ratings on the Motorola-provided charger against the other USB chargers and they matched. Other USB devices even charged fine with the Motorola charger. But the Motorola phone only charges with Motorola’s “special” USB wart.

That is absolutely assinine.

My next phone will be a Neo 1973.