Thursday, December 31, 2009

Chomping in Emacs

While working on Ezbl, I came to the terrifying realization that Emacs doesn't have a chomp-like function to strip leading and trailing whitespace from a string. After some searching, I found a solution, but it was kind of ugly (specifying the whitespace characters exactly rather than using a character class), so I modified it a bit. Here is my result:
(defun chomp (str)
  "Chomp leading and tailing whitespace from STR.

Why doesn't Emacs have this built in?"
  (let ((s (if (symbolp str) (symbol-name str) str)))
      ;; Make the [:space:] class match newline.
      (with-syntax-table (copy-syntax-table)
        (modify-syntax-entry ?\n " ")
        (string-match "^[[:space:]]*\\(.*?\\)[[:space:]]*$" s)
        (match-string 1 s)))))
The magic is all in the regular expression, which eats up as much whitespace as possible from the beginning and end and returns whatever is left in between (because of the non-greedy "*?" operator). By default (or in the mode I was using), the newline character is not considered part of the whitespace class, so I add it to a temporary syntax table. Any other characters which should be considered whitespace could be added in the same way. Maybe this can be included in a future version of Emacs, since it is useful and not too complex.

Update: So about 3 minutes after feeling all smart and cool for posting this, I made a comment on the #emacs IRC channel and immediately got a response back pointing me to replace-regexp-in-string. That whole big (ish) function collapses down to

(replace-regexp-in-string "\\(^[[:space:]\\n]*\\|[[:space:]\\n]*$\\)" "" str)

So yeah, quite possibly too short to warrant its own function. Serves me right for being so high and mighty with my fancy syntax-table. Silly mortal, Emacs always knows better.

Saturday, December 19, 2009

Yo dawg

Yo dawg, I herd u liek Emacs, so I put an Emacs in ur Emacs so u can Emacs while u Emacs. For more information, check out my older Ezbl post. EDIT: See new Special Edition Emacs browser meme: Yo dawg, I herd u like Emacs in a browser, so I put a browser in ur Emacs so u can Emacs in ur browser in ur Emacs while u browse ur Emacs in ur browser.

Friday, December 18, 2009

The Mason's Dilemma

I just read Tim O'Reilly's Why Using ShopSavvy Might Not Be So Savvy, and have a few thoughts. The first of which is that I think that people will fail to see the connection between "Mason" and "Brick and Mortar" stores, which is largely a result of a sub-par title. "The Brick and Mortar Store's Dilemma" doesn't quite have the same ring, though. Bad metaphors aside, the gist of O'Reilly's article is that by browsing for an item in person and then buying it online, you incur all of the costs of running a retail business (rent, display items, employees, etc.) without providing them with the revenue (from the purchase), making it a very unstable system. He is absolutely right about this, and it is a big problem for physical and especially small, local retailers. I don't dispute his predictions of how this will play out, but I find myself leaning towards a few different conclusions about what we should do about it.

Last Man Standing

First of all, he states:
But what happens once those mega-retailers are the last one standing? Prices are likely to go up.
I don't know about that. I'm definitely not a hardcore Laissez-faire kind of guy, and it really isn't the main point of his article, but I think that setting up an online retail site (or account on Amazon or Ebay) has such low overhead that the barriers to entry are too low for a monopolist to jack up prices. A local mega-store might be able to get away with it, but they would face the same competition from online retailers that the smaller shops do. Assuming that the mega-stores can't jack up prices, there is an argument to be made in favor of small local stores being pushed out by large chains. If goods like food, paper, cleaning supplies, etc. are cheaper, then people will have more money left over for more "useful" things that create better jobs than sales clerks. This is known as the "yay efficiency!" argument, and I think it has to be balanced with the "everyone in the town goes broke" argument.

We're Nihilists

To some extent, I can't help but think that there is a degree of futility in his proposal to "buy where you shop" in that it is going to be hard, if not impossible, to get a meaningful group of people to voluntarily spend more than they have to on the same item for an abstract notion of "preserving local business." It sounds pretty defeatist, but I don't think it is sustainable to tell people "consciously avoid going for the cheapest option, even if it is more convenient." There's also a huge, unresolved Tragedy of the Commons dilemma in that I might decide that there are enough other people buying from the local store that it is okay for me to buy the cheaper, online option. It's great for me if everyone is paying extra to support the local store and I can get the advantage of its services without having to pay the premium for its products, but if everyone does it, then we're right back where we started. Part of the problem seems to be the same one facing the organic food movement: people don't perceive organic food being that important, so will buy the cheaper, non-organic product. You have to convince people that organic (or local, or brick-and-mortar) food is valuable enough to spend extra money to get it. Changing people's consumption preferences is really hard, and is probably not a good strategy by itself. I don't have any stats to back it up, but it seems like organic food is having more success going the "it's healthier" and "it's tastier" route than the "it's better for the environment" route, since health and taste are already important attributes to people.

What to do, what to do?

So what do we do? One approach that is mentioned in the comments and discussed in detail by my hero Clay Shirky is the idea of decoupling the two functions of local stores, the browsing and the purchasing. Shirky describes this far better than I could, but bookstores have already started doing this, by adding coffee shops in the stores, they are becoming social gathering points first and retail outlets second. The analogue for more general retail outlets might be a business whose purpose was to show off and let people play with gadgets and things without the assumption that they were going to buy them there, possibly supported by charging a membership. People like to go shopping not so much for the "walking away with new stuff" aspect but more for the "look at and check out new things" part. Malls have invested a huge amount of time and money into making themselves "experiences," but they don't make their money off of the "experience" part. If physical stores were thought of more as advertisements that you can go to rather than places to purchase things, they might have a better prospect. There are certainly issues with this approach, not the least of which is whether it is really viable, since the cost of setting up a physical store might be too high to be recouped by a reasonable membership fee. There are a thousand things that could fail, but another thousand that could succeed. Now is definitely a time for crazy ideas and experimentation, but I don't think the solution is to try to fight against the prevailing incentives.

Tuesday, December 8, 2009

Ezbl 0.3!!

After many long hours, I have finally reached a new milestone in the development of Ezbl! Check out the user-visible part of the changelog for version 0.3:

User-visible Changes:

* Automatically-resizing xwidgets

If you change the size of the window containing a browser window, Ezbl will
automatically resize the xwidget to fit the new size.

* Cookie handler

Ezbl is now able to honor cookie PUT and GET requests from Uzbl, using Uzbl's
(relatively) new "talk_to_socket" system. This means that it is not having to
spawn a Python process for each and every cookie on every page you visit, but
is talking over a UNIX domain socket (which is fast).

This, combined with the new event system has enabled the following:

* No more script

Ezbl is now 100% Emacs Lisp, and does not need to launch external processes
(aside from Uzbl, of course :). This is largely achieved through the new
event-manager system, which lets Uzbl notify Ezbl of changes and events over
Standard Output, making for a very quick communication.

* Much faster performance

This is largely related to the event and cookie handlers, but it bears
repeating. In a few tests of loading, Ezbl would take about 30
seconds to finish loading the page. That time has now shrunk to just over 2.5
seconds. By using the event manager to let Uzbl notify us of changes, rather
than querying and polling, the amount of dead-time has decreased
dramatically. One commit in particular (d99f336) is responsible for the
majority of this speedup.

* New Emacs dependency

Uzbl uses a special kind of socket type (SOCK_SEQPACKET) for its cookie
handlers, and Emacs doesn't support such sockets (before 23.2, which I helped
to add), so you will need to build a custom version of Emacs, available here:

Note that you need the "xembed" branch of that repository.

Performance has increased by about 10-fold (from "dismal" to "pretty good"), and most of it is due to a single, one-line commit. It truly is the little things that make all the difference.

As an example, loading took about 30 seconds in the bad old version and it now takes just over 2.5 seconds to do the same thing, while remaining much more responsive during the whole load.

Anyway, it is getting to the point where it is starting to be actually useful (though without keyboard support, it still isn't ready to replace Firefox), which is really cool! The biggest limitation at this point is that it requires patches to Emacs that are not currently in any released version (though my SOCK_SEQPACKET patch is scheduled for 23.2!), so it isn't something you can just download and play with right away. I'm working with the people involved to try to get the relevant pieces in line so that experiencing the latest and greatest browsing system for Emacs isn't quite so big of a burden.

Once again, comments are always welcome, and definitely let me know if you have given it a try!

Thursday, December 3, 2009

Ezbl updates

It has been a while since I last talked about Ezbl, my Emacs interface for the excellent Uzbl browser.

I have moved far beyond simply being able to embed the browser in Emacs, and have added resizable windows, custom mode-line text, and most importantly, a cookie handler. The only major piece which still needs work is the keyboard input, since the only way to get text into Uzbl at this point is to paste it in with the mouse. I have some ideas on how to move forward; most involve wacky hacks with the DOM, so we'll see what turns up.

The next big push will be taking advantage of the new-ish Uzbl events system, so I don't have to do as much polling. Profiling Ezbl (with the built-in "elp" package) when loading and a few Twitter pages gives the following, rather uninspiring result:

Yes, it really is spending 77 seconds of CPU time on ezbl-sync-request, and yes, that is TERRIBLE. Luckily, I know how to move forward. I had been using a reasonably clever (but apparently horribly inefficient) system for making synchronous requests to Uzbl (for getting the value of a variable, for example). Emacs has a number of built-in ways of doing this, but I am thinking that with Uzbl's event system, I can just store the value of all the Uzbl variables within Emacs (there aren't that many) and update them on the Emacs side whenever they change. This way, I'm not having to do an expensive query to Uzbl each time I want to update the mode-line, for example.

In other news, I worked (if you can call it that, it was about 4 lines of braindead-simple code) on a patch to Emacs to add support for the SOCK_SEQPACKET socket type. Since Uzbl uses SOCK_SEQPACKET for its cookie sockets, having this work was essential for being able to communicate with Uzbl (without having to patch Uzbl, which I wasn't keen on doing). There are plenty of problems with the SOCK_SEQPACKET patch to Emacs, not the least of which is that it only works for Unix domain sockets, and not actual network sockets. Bummer.

If anyone is comfortable (or at least competent, which excludes me) with SCTP, Emacs could use some expertise in adding SOCK_SEQPACKET over AF_INET sockets. It's a rarely-used but useful protocol, and it's a shame that there is something which exists that Emacs doesn't yet support :).

Stay tuned for more!