Background, implementation

I was inspired by recalling the story of the Automated Curse Generator, which seemed like something that would be interesting to implement for fun in my own time, as it did indeed turn out to be.  In short, the module examines input text and generates a graph with edges weighted based on character frequency, then traverses the graph to generate a word.

To generate the chains, the module builds a directed graph based on the seed text, where characters are linked to all the characters which are known to follow them, with edges weighted according to the percentage of all following characters any particular character consists of.  For example, the string "zezifadi r00lz dr" would generate the following graph, where the value of each edge is the probability of choosing that edge to leave the associated vertex:

Click for graphviz source code.

To generate a word, then, it can be as simple as starting at ' ' (the red node) and continuing to traverse the graph until another ' ' is encountered.  In reality, while that worked, it was awfully boring.  When seeded with some text in English, there was a disappointing number of short, boring (not to mention unpronounceable) words and far too few amusing longer ones.  Think 'ad' and 's' rather than 'throm'.

It was rather easy to generate more interesting words, however, by simply adding some word-length limits, defaulting to a minimum of 4 character and a maximum of 12, tunable via arguments to the word generation method of the map.  Rather than blindly following edges, as long as the word generated is shorter than the minimum, any chaining result of ' ' will be ignored.  When maximum length is reached, the word will be immediately terminated provided the current character has any connection to blank space.  If not, generation continues until such a connection is found.

What makes this so entertaining, I think, is its versatility.  Since word generation is based entirely on the character frequency statistics of the input text, it works for any language.  By extension, that means it could be easily be made to generate whole phrases in $(East-Asian language of your choice) by feeding it ideographs rather than Latin characters (ばかです (yes, I'm aware this is actually Kana)), or just nonsense that pronounces a lot like Simlish by putting in some other Simlish nonsense.

The script

Having implemented word generation in the module, it was reasonably short work to wrap the whole thing in a script so it could be invoked from the command line for great lulz.  Something like the following does a decent job of providing amusement by generating a word every 15 seconds.  For more fun, pipe the output into a speech synthesizer.

Tari@Kerwin ~ $ while markov.py; do sleep 15; done

Of course, before anything can be generated, a graph must be generated, which can be done via the -s option on the script or by invoking the addString method of MarkovMap.  Quick example:

Tari@Kerwin ~ $ # Add the given string to the current graph, or to a new one.
Tari@Kerwin ~ $ markov.py -s"String to seed with" -ffoo.pkl
IO error on foo.pkl, creating new map
seeeeed
Tari@Kerwin ~ $ # Add some Delmore Schwartz to the map via stdin
Tari@Kerwin ~ $ markov.py -ffoo.pkl -s- << EOF
> (This is the school in which we learn...)
>What is the self amid this  blaze?
>What am I now that I was then
>Which I shall suffer and act  again,
>The theodicy I wrote in my high school days
>Restored all  life from infancy,
>The children shouting are bright as they run
>(This  is the school in which they learn...)
>Ravished entirely in their  passing play!
>(...that time is the fire in which they burn.)
>EOF
idagheam
Tari@Kerwin ~ $ # Generate a word from the default graph in file markov.pkl
Tari@Kerwin ~ $ markov.py
awaike
Tari@Kerwin ~ $

Easy enough.  I've found that a Maori seed (via Project Gutenburg) makes for some of the more easily pronounced words, but any language will (mostly) generate words that are pronounceable via that language's pronunciation rules.

For seeding with non-Latin character sets, the script can take the -l or --lax option ('strict' keyword parameter to MarkovMap.addString()), which removes the restriction keeping graphed characters as only alphabetic.  The downside, then, is that everything in the input is mapped out, so you're much more likely to get garbage out unless the input is carefully sanitized of punctuation and such (GIGO, after all).

Code

Enough talk, I'm sure you just want to pick apart my code and play with nonsense words at this point.  Download link is below.  I'm providing the code under the Simplified BSD License so you're allowed to do nearly anything with it, I just ask that you credit me for it in some way if you reuse or redistribute it.

Download markov.py

Cherenkov glow in the Advanced Test Reactor

• While radioactive hunks of matter are often portrayed as glowing with a green tinge, we all know that's not actually true.. unless there's Cherenkov Radiation involved, as in many nuclear reactors- that's not green, though.
• Google have (for now) won the suit against them by Viacom regarding copyrighted content being uploaded to YouTube, which is good news for everyone except maybe Viacom.  It's still fun to read choice excerpts of correspondence involving all sorts of mudslinging in the case (warning: lots of curses).
• OpenStreetMap is a neat project to create free maps, similar to Google Maps, Bing Maps, etc.  Cool stuff, and all the map data is Creative Commons, meaning it could be used for any number of shiny projects.
• There might be life on Saturn's moon, Titan, observations courtesy of the NASA/ESA/ASI Cassini mission, which has been bouncing around the Saturnian system since mid-2004 after launch way back in 1997.  It's far from a sure thing, but it's really exciting that predictions of how life might work on Titan have been supported by observation.
• This study (PDF) of internet routing to previously unused blocks is quite interesting, especially the numerous SIP streams pointed at 1.1.1.1 (section 5.1).
• The EFF (kind of like the ACLU of internet, if you're not familiar with them) recently put out the HTTPS Everywhere extension for Firefox.  When it's this easy to lock down your web traffic, there's no reason not to.  What's your excuse?
• Huge things are cool.  Want to feel tiny?  Go ask Wikipedia about the local supercluster, then consider how tiny everything humanity knows is, relative to that.  When you're done scrabbling about in your own Total Perspective Vortex, consider epic timescales for extra kicks.  Yeah.. cosmology is awesome.

• Exasecond timescales are impressive, but then there's the depressing concept of the universe's heat death.  What if there were a way to reverse entropy?
• How did I not know this album exists?  There are some really good mixes on it..

..and that's several weeks of accumulated cool-things.  Enjoy.

From the project page:

PuTTYJL is a wrapper and patch for PuTTY written in C# for .NET 3.5 and Windows 7, adding support for the new Jump Lists, allowing you to create jump list entries for saved sessions in the registry and optionally just launch the wrapper to start a default session in PuTTY.

Get it here.

As the authors of the paper note, sequencing genomes is nothing new, but there's a gigantic leap between just knowing how something is made and being able to make it yourself.  Although this modified strain of yeast has mostly stock genes from other yeast and just over a million base pairs, Wired Science notes that our ability to manufacture chunks of DNA has grown by around 100x in the last five years.  Following such a linear pattern, we would be able to build a human genome from scratch (~3 billion base pairs) within ten years.  From here, where can we go?  Anywhere.

Consider what living things do in nature.  Now take some of that variety and modify it a little to do something more useful.  Say, design an enzyme allowing yeast to break down oil from spills and removing any other metabolic pathways.  You suddenly have a bacterium which eats oil spills, then the colony dies when the oil goes away.

Sure, something like that is a ways off; we don't have anywhere the necessary knowledge of the biochemistry involved in such a thing (or do we..?  I could be entirely wrong).  Proteins are amazingly complex molecules, and their assembly/folding is rather poorly understood at best.  However, give it a while, and we could begin to do radical things within the framework of living things.  Say, custom-designed viruses to patch our genomes.  Literally, life hacking.

This is simply incredible stuff, and it's the first step toward the singularity, IMHO.  More thoughts on that in the coming days.

Look, it's CLI-tan.

..or don't (if you did, good).

Ubunchu is.. interesting.  I've lately taken to ignoring most things that so much as mention UbunchuUbuntu (as my StumbleUpon history will attest to), but it's good to know that there's still plenty of sense, if you know where to look.  Not everyone is spewing nonsense from their nostrils, evidently.

The truth is over there

There's a pretty impressive amount of truth lurking in Ubunchu under the silliness and not-quite-OS-tan levels of moe.  Basically every opinion voiced in the manga is accurate, IMHO.  In my mind, there's a place for each OS- each has is strengths, and certain weaknesses- what you get from any system is a combination of what it can offer well and what you put in.

Ubuntu, for example, is a very newbie-friendly Linux distro, and it's very good at being free and (generally) easy to use.  Try to control it too much, and it might break on you- that's just how the game works.

Arch, my Linux distro of choice, is quite different- it's one that expects you, the user, to go poking around and configure things yourself.  Arch is great at being configured to exactly fit your needs, provided you're willing to take the time to learn things for yourself (do eeeet, I say).

Windows is good at being itself- not free, but generally worth the price for those who know what they're doing.  I'll freely admit that Windows tends to be overkill and even something of a liability (what with malware and all) for uninformed users, but I feel that power users can get a lot out of Windows with excellent dev tools like Visual Studio, a huge multitude of games, and the multiple ways to do just about anything.  Say what you want about Microsoft, but their developer support is superb.

OSX?  Well, it's good at being simple to use and not free.  I won't say much on that since I generally make a point of ignoring Apple products, but I hear such systems are well-liked in the creative community.  My main complaint is that it generally only gives you one way to do whatever it is you might want to do, which is rather painful for someone like me, who likes to poke around in things.

Bottom line

Operating systems bring a variety of benefits to the table, and it's up to you, the user, to decide which one(s) fit your computer usage style best.  Now if you'll excuse me, I'm off to SSH into my server to check on my CLI IRC client. ( ^‐^)_

Oh, and I want a sysadmin's club.

Raw Data

Clock speeds are in MHz, TDP in Watts, and cost is price in USD at newegg as of 5/3/2010.  Processors with SMT (hyperthreading) are noted in the Cores column.

Performance Charts

I started by charting the benchmark scores for each processor in the table in Cinebench R10 and Crysis, using benchmark results from bit-tech.  The Crysis results are very limited since Bit-Tech's benchmark settings varied.  I used 1680x1050, all settings on High, no AA or AF.

[Note: images are no longer on the server.  If anyone really wants them back up, I'll regenerate them, but otherwise I won't bother.  Let me know if you want them.]

Cost Comparison

Finally, I compared the benchmark results and cost, charting how each processor scored when the cost was divided by benchmark scores.

Analysis

AMD's processors generally offer better price-to-performance ratios, but Intel wins out in terms of raw performance (I didn't include Intel's 6-core processor here, so the 1090T still tops the charts, though).

Intel's newer i5 and i7 cores performed noticeably better in Crysis, and perform better per-core in general.  Given the low cost-performance ratio and good raw performance numbers, the Core i5-750 seems to be the best choice for processors in my price range at the moment.  Even better, the TDP of the i5-750 is a full 30 Watts below that of the comparable AMD processors, all of which weigh in at 125 W.

The one factor that remains to be seen in this cost analysis is motherboards, but I think it's pretty safe to assume that costs for AM3 and LGA1156 boards are similar.

Generally

Gurren Lagann is the product of Gainax, the same studio (notably) behind Neon Genesis Evangelion, but that doesn't really mean anything- the two are completely different, even in animation style, which took a bit of getting used to.  It's a mecha, so you've got plenty of robots stomping around, explosions, and post-modern themes.  Most importantly, however, Gurren Lagann is loaded with silliness.  To get a feel for exactly how silly, let's just say that at times there are multiple levels of mecha-piloting-mecha recursion.

Partial cast.

"Your drill will pierce the heavens"

See what I did there?  That's basically the point of the entire thing, and it's the source of most of the silliness.  There's a very good reason for it all, but whenever something happens that makes no sense, it can be justified because "drills are magic".

The setting

It's some undisclosed alternate reality or future, where it seems all of humanity lives underground in small villages, and it's been that way long enough that it seems like the way things have always been.  Simon the Digger lives in one such village, working to expand the village, but it's not really why he digs.  He digs to find treasure.  In addition to Simon, we have Kamina, a bit of a troublemaker, who wishes to escape the conservative village elder and escape to the surface, dragging Simon along, piercing the heavens to freedom.  It's a pattern that gets repeated multiple times throughout the 27-episode run: fighting for freedom.

Bottom Line

Gurren Lagann is silly, fun, filled with explosions, and totally worth watching.

Oh, and how's this for a tidbit?  40% of the animation budget for the entire series was spent on the last 5 episodes.  The sheer insanity is.. mind-blasting.

I agree.

Comments, questions, suggestions are all welcome, and enjoy the new setup.

All's quiet on the Western front, it's a balmy 70º in Houghton for some reason.