2008-12-17 03:31:35 Audio Scraping in F-Script
If you live in the Cocoa frameworks as much as I do, you may just enjoy its conceptual approach to problems. One reason why I use F-Script is that there are lots of non-Cocoa projects that I want to use this technique in. Below is a quick example. I was looking at a page full of sound files from 2001 that I wanted to download. Various browsers and utilities allow you scrape such things from a site, but I wanted to just pull up a terminal and grab 'em quickly. And it was very straight forward. One could write a script for future use, but this is all I had to do in a Terminal window:
( Terminal ) ✂
  1  % cd Downloads
  2  % mkdir 2001
  3  % fscript
  4  F-Script> url := NSURL URLWithString:'http://www.palantir.net/2001/sounds.html'
  5  F-Script> prefix := url absoluteString stringByDeletingLastPathComponent
  6  F-Script> html := NSString stringWithContentsOfURL:url
  7  F-Script> wavs :=(html allCaptures:'tma1/wav/[a-z]*.wav') group
  8  F-Script> wavURL := [ :wav | NSURL URLWithString:(prefix stringByAppendingPathComponent:wav) ]
  9  F-Script> data := [ :wav | NSData dataWithContentsOfURL:(wavURL value:wav) ]
 10  F-Script> [ :wav | (data value:wav) writeToFile:((wav pathComponents) lastObject) atomically:YES] value:@wavs

Granted, there was more typing than that, but these were the only lines one needs in retrospect for this specific task. For instance, I had to check out the value of html to see what pattern was needed to define wavs. And I went back and changed the way I did things to make them more proper, and tested it, but what's shown above is ultimately an elegant way to proceed (and, for Cocoa types at least, intuitive).

In script form, it would roughly be:
( fscript ) ✂
  1  #!/usr/bin/fscript
  2  url := NSURL URLWithString:'http://www.palantir.net/2001/sounds.html'.
  3  prefix := url absoluteString stringByDeletingLastPathComponent.
  4  html := NSString stringWithContentsOfURL:url.
  5  wavs :=(html allCaptures:'tma1/wav/[a-z]*.wav') group.
  6  [ :wav |
  7      data := NSData dataWithContentsOfURL:(NSURL URLWithString:(prefix stringByAppendingPathComponent:wav)).
  8      data writeToFile:((wav pathComponents) lastObject) atomically:YES] value:@wavs.
  9  ] value:@wavs.

Which points out interesting stylistic differences between the way one would do things in the Terminal vs how one would write it up in a script. Of course, in a more realistic script there remains a need for the user to input both the url string and the search regex, which may be why this may not seem worth it to make a script out of this since it comes up so infrequently.

Thanks always to Philippe and Andrew for making life more fun.
RSS Feed for this page
Leave a comment