JB Journals
My Adventures in Byte Land!
PTA: The Pigeon Theory
May 20th
I guess many of you have heard about the recent controversy surrounding the ‘Draw Muhammad (P.B.U.H) Day’. Talk about holocaust and the whole world weeps. Insult our Beloved Prophet (P.B.U.H) and the world says ‘Free Speech’.
But that’s not why I’m writing this. My post is purely about the technical aspects. Apparently, PTA has blocked both Facebook and the many sites that promote this despicable event. And apparently for some reason, the powers that be also thought it necessary to block the Wikipedia entry for ‘Holocaust’.
Many of us in Pakistan have heard the story of a group of pigeons being attacked by a cat. The pigeons close their eyes thinking that if they don’t see the cat, the cat doesn’t see them either. The moral of the story is probably that ignoring problems doesn’t make them go away. But some people have apparently taken the story quite literally. I mean, blocking access to the sites from Pakistan isn’t going to make it go away, nor is it going to hurt Facebook that much either. All it will do is to weaken the credibility of Pakistan even more as a state run by fanatics (and quite the tech savvy stupid fanatics at that). The site is still online, I know because I used a proxy to access it. The Holocaust page is still there, I know because I just visited it using Googles cache.
All in all, the situation reminded me of the Cat & Pigeon story quite literally. Some guys from Pakistan have taken the initiative to take the site down using a Distributed Denial of Service attack by asking people to continuously ping the offending sites. For the interested, their site is located at: http://lemonbits.net/. They even made a small Windows Batch program that people can download that will relentlessly ping the offending sites, and at first I think it worked. But the PTA block effectively put a stop to that. All it does now is to flood the PTA monitoring server with requests for the page, and the PTA server has to handle all the millions of requests that generate from Pakistan for the sites in question.
In the end, all I’d like to say is that this definitely isn’t the way to handle such situations. We have tried this in the past many times, and almost never achieved anything (although we were able to take down Orkut once if I remember correctly, but even that was an unintentional by product of our block caused by DNS). In my opinion, the best way to handle this situation is by making an official complaint on the behalf of Pakistan. We are an important country, after all, we fight for the Americans against Taliban. We have a lot of leverage, we just don’t use it because that would mean that our politicians would have to get off their asses and move around.
And if that doesn’t work, LET LOOSE SOME HACKING. We’re pretty good at that, after all, according to recorded computer history, it was a Pakistani who created the first computer virus!!!
NEW INFO: This has gone too far. Now PTA has blocked YouTube 
. I wonder what’s next, GOOGLE????
YouTube Downloader: The Demise
May 19th
Finally, I’ve found a better alternative. Although still available at the time that I wrote my own version of the YouTube Downloader, I recently tried using it and found that YouTube had made some changes in the way YouTube videos are served from their servers. That practically made my own downloader pretty much useless, so I went in search for an alternative. And I found youtube-dl, a Python program that can be used to download videos off YouTube via the command line. Exactly what I wanted when I created my own YouTube downloader, except that youtube-dl is WAY better. I mean, the list of features is pretty darn impressive:
- Automatic detection of playlists via the URL you give it. Download of complete playlist
- Templates for easy naming of each downloaded file. For instance, you could name each downloaded file according to the title of the video
- Export of the links of the videos instead of downloading
- Built-in intelligent downloader with interrupted download resuming capability
So, I have finally given up on my own project and shifted to using youtube-dl. If you wish to download it, get it from:
http://bitbucket.org/rg3/youtube-dl/src/tip/youtube-dl
Instructions for using it can be found on the youtube-dl wiki. All in all, it’s a VERY impressive piece of work and I suggest you use it for all your YouTube video downloading needs.
7 Years with Open Source Software: Part 1
May 5th
Ubuntu 10.04 came out about a week ago, and it got me thinking about my personal journey with Open Source software. So I decided to write about how I got into open source, how I watched it grow, and how I use it now, with a list of some of my favorite open source software. I also include some of my wishes for open source software, and some of the problems I have encountered with it.
Please note, I started this article with fair warning that this is going to be purely about MY experiences with open source. I know that many of the problems I point out wouldn’t have been encountered by most of you, and many of my assumptions will be false, and I may make wrong conclusions. Many of the shortcomings of open source I talk about here may just be due to my own stupidity, so please, before you tell me to RTFM (read the freaking manual) or start yelling at me about how everything I have written is crap, please remember, I’m just writing about my OWN experiences. Yours may be different.
When I first started writing, I planned just one post, with all my thoughts about open source. But once I started, I found that writing that much in just one post would be a gruesome punishment for anyone reading it (come on, you really want to read that much from a guy who can’t even vouch for his own sanity 
), so I decided to make this into a multipart series. I hope to post every week or so and cover most of my major encounters with open source software. If you like this article, please do comment. Also be sure to point out if you find any mistakes, I’m the first one to admit I’m not perfect.
I’m amazed when I see how far Open Source Software has come.
The Start:
Well, I don’t remember exactly, but I was introduced to Open Source software around 2003. It wasn’t so much as an introduction to open source than a review of Linux that got me interested. That year, one of the issues of the monthly I.T. magazine Spider did a small piece on Linux, how it was better than Windows due to it’s extreme customizability and security. At that time, I was nearing completion of my O’ Levels and had just completed my course project for the Computer Sciences course. During the 5 months that I took developing it (another story worth repeating some other time), I had to reinstall Windows (I think XP was the prevalent version at that time) at least 4 times, partly because of virus infections (I didn’t have the sense to use anti-virus back then, as I had a rather old Pentium 3 and anti-virus made it crawl to a halt) and partly due to Windows becoming too sluggish to use due to the plethora of software I usually installed after every reinstall. I guess we all know the result of mixing Windows with real world computing:
So there I was, having just read about the stability of Linux. Add to that the fond memories I had of reinstalling Windows over and over again, I was practically jumping with the excitement on having found an alternative.
rename: Linux Batch Renaming Utility
May 2nd
Imagine this scenario: You just got back from a picnic, and you have hundreds of photos of the trip in your digital camera. You want to copy those photos to you Pictures folder, but now you run into a small inconvenience. All the photos from you camera are name something like Image_0001.jpg, Image_0002.jpg and so on for all of the hundreds of beautiful moments that you captured, and that ticks you. You want to name you photos according to the occasion, say something like Picnic_2010_Img_01.jpg, and you don’t want to go through the unimaginable pain of having to rename all those photos by hand. So, what do you do?
To give you a head start, here’s what you would write on the console to rename those files:
rename -v 's/^Image_(\d*)\.jpg$/Picnic_2010_Img_$1.jpg/' *.jpgAs you read along this tutorial, I will explain each part of this command, so that by the end of it, you will be able to modify it for your own needs.
Meet rename, a small Perl script/program available with most Linux Distros (including Debian, Ubuntu and Fedora). This small program can help a lot when you want to batch rename files. You might already have used similar programs in Windows or even Linux, and most of those programs would have had user interfaces with pretty pictures and buttons for you to click and all. They may even have wizards to help you specify exactly which files you wish to rename, but if you’re anything like me, you’d rather use something that gives you full control over how your files get renamed without going through the hassle of setting loads of options before you can start, and you’d want to do that quickly. That is how I came across rename, and ever since I learned about it, I’ve been using it for batch renaming, ditching all those other high end graphical programs, simply because rename is a LOT faster to use in most situations.
First, a warning for the weak hearted. If doing things by typing away at the console isn’t your thing, I suggest you stop reading, because rename is a console utility. It doesn’t have any graphical interface, nor any wizards to help you along the way. It simply does what you tell it to. That of course means that you have to spend some time learning it before actually use it. But, believe me, any time spent learning about the incredible power of rename is time well spent, and in the long run, you end up saving loads of time simply by skipping all the unnecessary configuration that you have to go through when using other batch renaming programs.
Now, rename is simply a Perl script, and like most Perl programs that do text processing tasks (which renaming is, since you’re just replacing some text which happens to be the name of a file), rename uses Perl regular expressions (regex) to do it’s magic. According to the man entry for rename:
“rename” renames the filenames supplied according to the rule specified as the first argument. The perlexpr argument is a Perl expression which is expected to modify the $_ string in Perl for at least some of the filenames specified.
What this says is that you give rename a list of files and a rule (written using Perl regex) to apply to the names of those files. rename applies the rule to each of the file name, and if the rule ends up changing the name, rename simply renames (no pun intended) the file. That may sound complicated, but trust me, once you know the basics, it becomes really simple.
Wateen Auto Login (the death)
Apr 19th
Apparently, most of the traffic on my blog comes from people searching for a Wateen auto login program, which is kind of sad in some ways as I actually wanted this to be a blog about my programming activities.
But anyways, I just have to tell all of you that I have left Wateen for PTCL EVO for quite some time now, and I am not going to maintain my Auto Login program (not that it was in any way maintained before ). However, for those of you poor souls that are still stuck with Wateen, I’m going to suggest an alternative for the Auto Login program.
- Leave Wateen!!! Seriously, all I got were headaches from their service. No connection for hours, and when it did connect, I got speeds of around 10KB/s…. Yeah, that right, TEN KILOBYTES PER SECOND!!!! EVO actually rocks a lot more, and without me having to call the helpline every other day.
- If you’re still here, you apparently haven’t left Wateen yet, and still need an auto connector. Before I created my own, I did search around a bit for something that I could use. I only created my own because my needs were very specific (to automatically start/stop my bit torrent downloads according to the status of my Wateen connection and that too on my laptop only). I have come across an application that would be perfect for most people out there:
Wateen WiMax Keep Alive
Its built in Java, and so should run on almost any Operating System (Windows XP/Vista/7, Ubuntu, MacOS). Also, the interface is quite simple to use, so it should fulfill the need of most people needing an auto connector. To use it, simply download the zip file (if you’re lazy, a direct link to the zip is here), unzip it in a empty folder, and just run “Wateen WiMax.jar”. Once the application window is shown, go to File->Preferences and fill in your username and password details. Then click the OK button and you’re all set to go. As long as the program is running, you Wateen connection will connect automatically every time it gets disconnected. And if you set the program to start automatically, it will also connect your connection when the computer starts up.
Hopefully, this will be useful to someone out there.
Long Time, No See!!!
Feb 10th
I admit, it’s been a REALLY long time since I last made a post about anything. And boy do I have a lot to discuss. The most important thing I guess would be that I finally got my own domain name AND hosting!!!! YAY for me…
It really pays off to have your own hosting and domain. The domain part is good cause I can now get my site listed on google according to my own preferences (by using the googles webmaster tools), have better visitor records (using google analytics), and just have a lot more fun than I could with a free domain. The hosting part is the best, I can use custom themes with wordpress (something that I really wanted at www.wordpress.com), install my own plugins, have ftp access to my blog, and a hell of a lot more that I actually don’t even know how to describe…
If the above sounded like the rants of a madman, rest assured, it is. I’M VERY EXCITED TO HAVE MY OWN DOMAIN NAME!!!!!
FunMaza Downloader Revised
Dec 2nd
Finally settled on a first design draft for the FunMaza Downloader program that I have been going on about for some time. For anyone interested in it, the draft is available on PasteBin here.
Unique Links Extractor 2: A Python Exercise
Nov 19th
A while back, I learned Perl and decided to program a link extractor in it. While I was successful, as my other post tells, I was by no means satisfied. There was always a lingering feeling of confusion whenever I programmed in Perl. Coming from a C/C++ programming background, I feel very uncomfortable programming in Perl. Maybe its the way the language is designed or perhaps I just wasn’t using it right, but I never quite got the sense of ease while programming in Perl. I would always write a piece of code and then coming back to it after a few hours, I would always be confused about how things worked. Its not that the language of Perl is inherently confusing, to the contrary, It’s quite easy to use, but I never felt as good using it as I did C. Somehow, C has something beautiful that I am attracted to (I know, I’m a NERD…). Perhaps it the syntax of C/C++ that makes it look more like a REAL programming language, or maybe its just that after 5 years of use, I can actually THINK in C. I really think that its the 5 years of use, but whatever the case may be, I prefer Perl only when making small programs that have some really heavy reliance on Regular Expressions, other than that, I just stick to C. More >
YouTube Downloader
Oct 7th
This post is about a dead project. If you’re looking for a YouTube downloader, check out my latest post.
OK, so I finally finished 1 project that I was thinking about for some time now (around 2 weeks). It is a collection of Perl scripts that can be used to grab the download links for YouTube videos from a YouTube playlist page.
I was looking for such a thing to download a couple of Dramas (Pakistanis TV Shows (sort of)), and not finding anything appropriate, I decided to create one myself. There are a total of three scripts that can be used together to grab the video download links (links to actually download the videos, not just play them in a browser) of all the videos in a YouTube playlist.
The first script extracts the links to watch the videos (in a browser) from the playlist page. It follows all the ‘Next’ links at the bottom of the page, so if you provide it with the link of the first page in a playlist, it will extract links for all the videos in that playlist.
The second script extracts the download links from the links that the last script extracted (the ones used to view the video in a browser).
The third script takes the output of the last one and converts it into an HTML page, with each video title being a link to download that video. I created this so that I could use the DownThemAll Firefox extension to easily download the videos, rather than create my own script to do so, which would be really complex since it would need to keep track of what was downloaded, what remained, and so on.
So, I have, finally, created my first program based on the Unix philosophy, small programs that do one thing well working together.
On a side note, I just discovered that Pastbin is a wonderful site to post my code online at (it even has syntax highlighting!!! WOO!!!). So, from now on, I’m gonna be posting all by source code at pastbin. The links to the scripts mentioned above are just below. Enjoy!!!
1) Playlist Parser (to get video watch links from a playlist page)
2) Video Link Grabber (to get the download link from a video watch page)
3) HTML Links Generator (to generate HTML page from the output of the number 2 script)
To use the script, run the first one; giving the link of the playlist page as the only argument. Save the output to a file. Then run the second script, giving it the path to the file produced in the last step. Save the output to a file as well. Finally, run the last script, giving it the path to the file produced in the last step. Save this output to a file as well. Now, use your favorite downloader to download the files from the links in the HTML file (I use the DownThemAll extension of Firefox. It is just awesome, giving me a great deal of control over what to download, where to save it to, and how to name each file).
Fun Maza Downloader: Back to the Basics
Sep 6th
OK, so first of all, I’ve been absent for quite a while. I was actually busy with papers and stuff, and then I was just too darn lazy to get a new post. SORRY!!!!
So, here goes. If you’ve read my blog, you’ll know that I was working on a Perl script to download all the songs of funmaza.com. I did that, made a script, ran it, and it actually downloaded stuff…
But then, I didn’t like the end result. There was no way to stop the downloading, it didn’t keep an accurate log of what it downloaded and what was left to download, and honestly, the code was a HUGE mess. So, as I always do, I deleted the entire thing (believe me, I’ve done it a hundred times with around 5 websites, 10 different programs, 3 blogs, and well, u get the drift… ).
So now, I’ve decided to remake the entire thing. Yesterday, I came across a new browser; Uzbl. Its a browser thats based on the Unix philosophy, “Write programs that do one thing and do it well. Write programs to work together. Write programs to handle text streams, because that is a universal interface.”
This got me thinking, and now I’ve decided that thats the way I’m gonna make my downloader. A number of small programs, each doing only one task, that work together using text streams.
Wish me Luck…