The Storyteller

Entries categorized as ‘PHP’

This is my last week in TrippertLabs

June 21, 2008 · 38 Comments

I am leaving TrippertLabs by the end of this month. It is kinda painful leaving something which I have developed actively as a part of management entities. And TrippertLabs become a big hit here in Bangladesh for PHP devs. In the past one year in TL I have managed to set it up with 4 out of 5 ZCEs in Bangladesh and in total of 8 awesomely skilled PHP devs, 4 outstanding game developers and animators, 3 QAs and one administrator here in this local facility . TrippertLabs BD became a highly equipped development center for developing high traffic and game based social network applications (for Facebook, MySpace, Bebo and OpenSocial) along with it’s other wings in Indonesia, Pakistan, India, Germany and USA.

So this is end of a one year journey with TrippertLabs here in Bangladesh.

I am planning for a one month vacation and then I will start looking for a job again.

Categories: Me - Myself · PHP · Trippert
Tagged: ,

Look Ma, everyone’s computing out there for me!

June 6, 2008 · 4 Comments

SETI@Home is probably the greatest example of low cost distributed computing which become a big hit. After their tremendous success, many others over there started following the same strategy and used the power of distributed computing for other purposes like cancer research. In this article I will show you how we can use the same power at almost zero cost, and specially for your web applications.

As I am currently working on building an open source version of FriendFeed (not targetted to be an alternative, coz those people at FriendFeed done their job really well) and scaling such a huge load effectively at low cost, so I will mainly talk more about FriendFeed through out this blog post and use it as an example for my proposal.

If you consider FriendFeed as a repository of feed URLs, a lot of peoples and how they are related to each other, you can assume how big it is or it could be in near future. And scaling such a service would cost numbers of sleepless nights of many developers out there. So in basic, lets focus where actually the problem is and how we can introduce distributed computing.

Beside optimizing database to serve huge sets of data, one of the main problems of such a service has to parse millions of feeds in a regular interval. If we want to bear all the loads on your server, fine, if you can afford. But what about some low cost solutions. Lets consider a simple scenario, if your application has one million of users and each of them browse your application for 10 minutes a day, you really have 10 millions of computational power just wasting over there, in lanes and by lanes of internet - heh heh. So lets make use of such an incredible CPU power. All you have to do let the visitors machine do some calculations for you and free your server from gigantic load.

When the users of your application and relation among them are stored in your database, you can easily find out the first degree and second degree friends of a specific user. If you don’t know what does that mean, its simple, If A is a friend of B and C is a friend of A, then A is B’s first degree friend and C is B’s second degree friend. For a huge social network, it may look like the following one when you visualize the relationship


image courtesy: http://prblog.typepad.com

Now what we want to do is when B is visiting our application we want to parse most of his/her second degree friends in client using his browser. So while generating the page for B, we will supply him a bunch of feed URLs, a hash of their last known update time or hash of the latest item of each of these corresponding feeds, and a javascript based parser (for example Google’s ajax Feed API would do fine) script. Now, while B is browsing our application we will parse those feeds of his second degree friends using javascript without bothering him for a single second and post back the parsed contents (after checking against the hash for really updated contents) to a server side script which will then blindly (not totally, after some validation or authentication for sure) insert those result to our database. Now when A comes and visit his page (A is C’s first degree friend), he will get all the latest result from C’s feeds as B has already done the parsing job for us and we have those latest result from C’s feeds stored in our Database.

There are definitely more challenge than it is explained here, like what if a person is second degree friend of a multiple user. In such cases as we supplied last update time of these feeds while generating a page, we can calculate from server side which feeds we really want to parse.

And moreover, we can add more check for our javascript parser than just blindly parse those feeds. We can download a chunk of RSS or Atom feeds (using a proxy script developped using curl range options) and read just up to the latest update time and extract the time using simple reg ex or string functions instead of downloading full feed data. Now if we know that a script is not uptodate just by downloading 1-2 killobyte of data instead of downloading full feed, parsing the xml data, it would save us more computing resources for performing other jobs.

But of course, you cannot live completely on your client’s machine for parsing all your data. You must have several cron’d scripts to parse left overs or other feeds at server side. But what I am saying is that with little help of javascript you can make use of such a tremendous distributed computing power in your application, all at almost no cost.

I will come with example code once I am done developing my open source clone of FriendFeed and then I am sure, you will find it was worth writing a blog post about.

Distributed Computing
image courtesy: http://www.naccq.ac.nz/bacit/0203/2004Caukill_OffPeakGrid.htm

Have a nice RnDing time. :)

Categories: OpenSource · PHP · Tricks · idea
Tagged: , , ,

Building services like FriendFeed using PHP - Part2

June 3, 2008 · 6 Comments

Following the first installment in this series, here is the second part. In this part I will focus mainly on Bookmarking and News services supported by FriendFeed . Here we go

Supported bookmarking services by FriendFeed
1. Del.icio.us
2. Furl
3. Google shared stuffs
4. Mag.nolia
5. Stumbleupon

Except google shared stufss, all of the rests require just an username to generate the access point for retrieving user’s bookmarked items. And for the google shared stuffs, it requires the fully functional url of the feed available from your google bookmark service.

Access points

Del.icio.us

AP: http://feeds.delicious.com/rss/<user name>
example: http://feeds.delicious.com/rss/anduh


Furl

AP: http://rss.furl.net/member/<user name>.rss;
example: http://rss.furl.net/member/pigge.rss


Google Shared Stuffs
you can find your google shared stuff url from http://www.google.com/s2/sharing/stuff

example: http://www.google.com/s2/sharing/stuff?user=110703083499686157981&output=rss


Ma.gnolia

AP: http://ma.gnolia.com/rss/lite/people/<user name>.rss;
example: http://ma.gnolia.com/rss/lite/people/gerryquach


Stumbleupon
Whoops, Double Whoops, Tripple Whoops. It took quite a time for me to find the feed url. I dont know why it is kept so “SECRET” - LOL

AP: http://www.stumbleupon.com/syndicate.php?stumbler=<user name>.rss;
example: http://www.stumbleupon.com/syndicate.php?stumbler=jd001


Supported news services by FriendFeed
1. Digg
2. Google Reader
3. Mixx
4. Reddit

Here are the access points


Digg

AP: http://digg.com/users/<user name>/history.rss;
example: http://digg.com/users/msaleem/history.rss


Google Reader

You can find your shared item’s feed url here http://www.google.com/reader/view/user/-/state/com.google/broadcast


Mixx

Unfortunately during the time of writing this article, Mixx was napping - here is the screenshot. Once they are awake, I will update this section :)


Reddit

AP: http://www.reddit.com/user/<user name>/.rss;
example: http://www.reddit.com/user/jack_alexander/.rss

In next installation I will focus on scaling such a huge load successfully. Hope that will be interesting to many of you. Following installation will focus again on the access points.

Categories: PHP · Social Networking · howto · idea
Tagged: , , ,

Building services like FriendFeed using PHP - Part 1

May 31, 2008 · 20 Comments

Friendfeed is an excellent life streaming service aggregating all your feeds from different service providers, compile them together, build a social network among your known people and finally deliver all these feeds as a mashup. As a result you can immediately track activities of your friends on all the different service they use (like jaiku, twitter, flickr) and stay up-to-date. Hats off to the developers of FriendFeed for delivering such a nice application to us.

In this blog post I will try to focus on how to develop such a service like Friendfeed using PHP and JS and how to scale such a huge load successfully. Previously I’d also written another article focusing “How to develop services like SpringLoops using PHP“. The complete article will be delivered in multiple installments and in current installment, I will focus on Photo sharing services used by friendfeed.

Area to focus
1. Services to be supported
2. Update discovery
3. Parsing the feeds
4. Storing the data
5. Scaling

FriendFeed parses feeds from a total of 35 popular services from around the web ;) - lets have a look at them one by one and I will give you idea about the access points of these services. Though Not all of them are feed, so I will try to focus those services as well to give you a basic idea.

Service 1 : Flickr
Usable PHP Library: PHPFlickr

Flickr is an excellent photo sharing services under the banner of Yahoo. Friendfeed parses all the photo from a user and also the list of his/her favorite photos. Given an username you can easily find the Flickr ID like (76536911@N00)of that user and once you got it, here is the access points for list of public photos and favorites

List of photos
http://api.flickr.com/services/feeds/photos_public.gne?id=<user’s flickr id>&lang=en-us&format=rss_200
example: http://api.flickr.com/services/feeds/photos_public.gne?id=76536911@N00&lang=en-us&format=rss_200

List of favorites
http://api.flickr.com/services/feeds/photos_faves.gne?nsid=<user’s flickr id>&lang=en-us&format=rss_200
example:http://api.flickr.com/services/feeds/photos_faves.gne?nsid=76536911@N00&lang=en-us&format=rss_200

Service 2: Picasa Web Album
Picasa is another photo sharing service under the banner of the Giant, Google. Given a picasa user name , here are the access point of user’s public photos

List of public photos
http://picasaweb.google.com/data/feed/base/user/<picasa user name>?kind=album&alt=rss&hl=en_US&access=public
example:http://picasaweb.google.com/data/feed/base/user/countdraculla?kind=album&alt=rss&hl=en_US&access=public

Service 3: Smugmug
Here comes another photo sharing service where users can buy and sell their photos. Given a smugmug user name you can find the feed of this user’s photos

List of public photos
http://<smugmug user name>.smugmug.com/hack/feed.mg?Type=nickname&Data=Ifocus&format=rss200
example:http://ifocus.smugmug.com/hack/feed.mg?Type=nickname&Data=Ifocus&format=rss200

Service 4: Zoomr
Zoomr is another photo sharing service started in late 2005 by Kristopher Tate as a place for him to share photos with his friends and later became a public services, is currently supported by FriendFeed. And here we go with the access points :)

List of public photos
http://www.zooomr.com/services/feeds/public_photos/?id=<zoomr user name>&format=rss_200
example:http://www.zooomr.com/services/feeds/public_photos/?id=wolfmank&format=rss_200

In next part, I will highlight the access points of supported bookmark services and news services supported by FriendFeed :) Have a nice weekend.

Categories: PHP · Social Networking · howto · idea
Tagged: ,

Setting up virtual host with mod rewrite enabled in Ubuntu Hardy Heron

May 29, 2008 · 14 Comments

Virtual host is really what the name says, a virtual environment over the top of your current web server to simulate a separate hosting environment. Using virtual host you can enable site specific features and keep your development environment totally separate from another one. For example, if you want to experiment a simple application with both mod_rewrite enabled or disabled, you can setup two virtual host with these different settings to take place. In this blog post I will show you how to set up virtual host in ubuntu hardy heron with mod_rewrite enabled.

Step 1: setup a virtual domain

open /etc/hosts and add a virtual domain with a specific local IP. In this file it contains ip and domain name separated by a space. You can also add the port using a colon with the IP. Lets assume that our virtual domain name is “ilove.php” - and It will listen to the ip “127.0.1.2″

so we will add the following line to our /etc/hosts file
127.0.1.2:80 ilove.php

now whenever you point to http://ilove.php - your browser will actually open http://127.0.1.2:80

Step 2: configure virtual host with apache
here we will configure our newly added virtual domain against apache as a virtual host, and did I forget to say, with mod_rewrite enabled :)

goto /etc/apache2/sites-available and create a file named “ilove.php” - I recommend to keep it the same name as your virtual domain.
sudo nano /etc/apache2/sites-available/ilove.php

write the following contents inside. but please note to create the appropriate directory before linking your virtual host with that, for example we’ve create a folder named “/home/<user name>/www/ilovephp” and linked that directory as my document root in the following configuration file.


<VirtualHost 127.0.1.2:80>
	ServerName ilove.php
	ServerAlias www.ilove.php
	ServerAdmin admin@ilove.php
	DocumentRoot /home/<user>/www/ilovephp
	<Directory /home/<user>/www/ilovephp>
		Options FollowSymLinks
		AllowOverride All
	</Directory>

</VirtualHost>

now create a symbolic link of this file to /etc/apache2/sites-enabled directory as “ilove.php”

sudo ln -s /etc/apache2/sites-available/ilove.php /etc/apache2/sites-enabled/ilove.php

Step 3: restart apache (or reload)
simple, either one of the followings
sudo /etc/init.d/apache2 restart

or

sudo /etc/init.d/apache2 reload

or

sudo a2ensite ilove.php

and you are done! now you can point your browser to http://ilove.php

Categories: PHP · ubuntu
Tagged: , , ,

I dont give you a damn, if

May 12, 2008 · 98 Comments

You came to an interview for “PHP Developer” and you said you dont know the name “Rasmus Lerdorf”.

You’ve used MySQL for a long time and didn’t even hear about “Michael Monty”

and though “jQuery” is your “favorite JS framework” for a long time, but “John Resig, who is he?”

shame on you


The reason behind writing this blog post is I’ve interviewed some candidates recently and They dont know the name of the developers of PHP, MySQL or almost any other libraries they make their living on. They dont even know the history of these tools (History might be too much and I surely am not a Jerk, but I want them to know the name of the developer behind this)

I don’t take my time to show them the door, I am pissed off.

Categories: PHP
Tagged:

Web scrapping in a smart way, making a “Today in History” object in PHP

April 29, 2008 · 5 Comments

There are thousands of services available on web who are presenting interesting as well as education information which you can really integrate in your web page or make a nice widget and let others use them seamlessly with their content delivery platforms. In this article I am going to show you how you can make a nice Today-in-History widget with the help of the data provided in Scopesys. You can use this code to make a nice widget or a trivia app or whatever. But before making your own scrappers from any services, please please please carefully note the copyright of that content. You shouldn’t violate copyright either way.

In this widget, we will strip the following content from the pages provided by scopesys and display them in different categories.
1. Today in history
2. Who’s born today
3. Who’s died today
4. Where is holiday today
5. Religious observance of today
6. Religious history of today

Lets go :D


<?php
//todayinhistory.php
error_reporting(0);
define("MARKER_START","<H3>On this day…</h3>");
define("MARKER_END","<BR><BR><HR><h3>Holidays</h3>");
define("BIRTHDAY_START","</font></center></center>");
define("BIRTHDAY_END","<HR> <br><H3>Deaths which occurred on ".date("F d").":</H3>");
define("DEATH_START","<HR> <br><H3>Deaths which occurred on ".date("F d").":</H3>");
define("DEATH_END","<HR><IMG align=left SRC=\"http://www.scopesys.com/flag.gif\">");
define("HOLIDAYS_START",’<i>Note: Some Holidays are only applicable on a given <b>"day of the week"</b></i><br> <br>’);
define("HOLIDAYS_END","<HR> <H3>Religious Observances</H3>");
define("RELIGIOUS_START","<HR> <H3>Religious Observances</H3>");
define("RELIGIOUS_END","<HR> <H3>Religious History </h3>");
define("RELHISTORY_START","<HR> <H3>Religious History </h3>");
define("RELHISTORY_END","<BR><BR><font color=red>");


echo "<h2>Today is ".Date("F d, Y")."</h2>";
$data = file_get_contents("http://www.scopesys.com/today");


if ($_GET['history']==’1′){
echo "<br/><h2 style=’color: green’ >Today in history</h2>";
$end = strpos($data,MARKER_END)-15;
$start = strpos($data,MARKER_START)+strlen(MARKER_START);
echo substr($data,$start,$end-$start);
}


if ($_GET['born']==’1′){
echo "<br/><h2 style=’color: green’ >Who’s born today</h2>";
$end = strpos($data,BIRTHDAY_END);
$start = strpos($data,BIRTHDAY_START)+strlen(BIRTHDAY_START);
echo substr($data,$start,$end-$start);
}


if ($_GET['died']==’1′){
echo "<br/><h2 style=’color: green’ >Who died today</h2>";
$end = strpos($data,DEATH_END);
$start = strpos($data,DEATH_START)+strlen(DEATH_START);
echo substr($data,$start,$end-$start);
}


if ($_GET['holiday']==’1′){
echo "<br/><h2 style=’color: green’ >Where is holiday today</h2>";
$end = strpos($data,HOLIDAYS_END);
$start = strpos($data,HOLIDAYS_START)+strlen(HOLIDAYS_START);
echo substr($data,$start,$end-$start);
}


if ($_GET['religious']==’1′){
echo "<br/><h2 style=’color: green’ >Religious observance</h2>";
$end = strpos($data,RELIGIOUS_END);
$start = strpos($data,RELIGIOUS_START)+strlen(RELIGIOUS_START);
echo substr($data,$start,$end-$start);
}


if ($_GET['relhistory']==’1′){
echo "<br/><h2 style=’color: green’ >Religious history</h2>";
$end = strpos($data,RELHISTORY_END);
$start = strpos($data,RELHISTORY_START)+strlen(RELHISTORY_START);
echo substr($data,$start,$end-$start);
}
?>

Now if you want to find who born today, point your browser to todayinhistory.php?born=1. Mashup Mashup Mashup, that is what many successful web app are doing these days. And sometime this is how data collection is done behind the scene :)

Writing this code was really enjoyable as getting root canal done in your teeth with a rusty drill (I forgot where I’ve read such a nice quote), heh heh. But I am sure, you will enjoy it more than that ;) - happy scrapping.

Categories: Cool Webapps · PHP · idea
Tagged: , , , ,

How to make your own springloops in PHP

April 27, 2008 · 11 Comments

Springloops is a nice code management service recently came into focus. It helps you to manage the code base of your application, monitor the commit and deploy the final version easily to another server. So if you are wondering how to build such a system and how it actually works, this article is for you.

Please note that I am neither way affiliated with Springloops nor any of it’s contacts. This article expresses completely my own opinion.

The primary obstacles of making such a service are
1. Managing the subversion repositories and users
2. Interaction with subversion repositories (Not same as option 1)
3. Payment gateways
4. Code browser
5. User friendliness
6. Scaling

Among these options, 4 and 6 are out of the scope of this article. There are thousands of article explaining those topics to you. So I am not going to talk about them. There are excellent libraries available to manage many popular payment gateways. Just google it and you are done. And about user friendliness and design, you can hire a consultant who’s specializing on this subject and get it done.

1. Managing the subversion repositories and users
In springloops or any other subversion hosting service, you need to do some basic shell scripting which is required for adding the subversion repositories dynamically and to add users in it. There are some excellent articles how you can do it with apache2. To avoid any traffic related issue, best practice will be hosting these repositories in another server, and using htaccess - just point your user’s svn URL to the exact url. Check out the following urls to find out how to host subversion repositories with apache. Linux should be the first choice, heh heh.

Reference:
1. Setting up subversion with apache in Ubuntu gutsy gibon
2. Setting up subversion with apache2
3. Setting up subversion with apache2 and DAV in debian
4. Host your open source projects in ubuntu in 3 easy steps

All you have to do is writing shell script to automate these steps and restart the apache2 demon once an user register his/her project with your service. You are done

2. Interaction with subversion repositories (Not same as option 1)
Now this is quite a challenging part and many of you are lost how to achieve this kind of functionality for such services. For PHP developers, you know there are an excellent repository of extensions named PECL and also repository of libraries name PEAR. In PECL there is an excellent extension which is cent percent appropriate for this work. Yup, I am talking about “SVN” extension. You have to install this extension to use it with your PHP code. Once you are done with installing it, now you got the complete power to interact with the subversion repositories hosted by your users. This extension is very rich and provides all the functionalities you need to interact. For details please point your browser to the appropriate section in PHP manual.

Reference
1. Installing PECL SVN extension for PHP in Ubuntu Gutsy Gibon
2. PECL svn package
3. SVN extension documentation

4. Code Browser
Another challenging part of the total setup. If you cannot provide excellent code browser which will clearly demonstrate the changes in code from different revisions, figure out the log and the notes made during each commit and finally, make note on it to show to your fellow team members.

Using svn_diff, svn_ls and svn_log you can clearly find out the difference of a file in two different revisions, the file and directory structure of the repository and the log of each commit. So basically once you got the file and directory structure of your repository for a revision, you can just traverse through it and display as a nice report using your PHP code. And when user will ask you to display the difference, you will find the difference using svn_diff which returns the difference as standard diff format. Now using regular expression (or whatever way) you will mark the lines which was changed and just display them highlighting using different colors to your user, as a nice report. And You can also browse the log of any revision using svn_log and show it to your user.

And last but not the least, you can create tags, branch and whatever addition by maintaining a local shadow working copy of your repository. Only the thing that you cannot do (or still I am thinking how-to) is merging, heh heh.

And when it comes about deploy the latest code base to your server, you can do it using svn_update where the path is the working directory in your server.

Reference
1. Standard Diff Format
2. svn_diff
3. svn_log
4. svn_ls
5. svn_add
6. svn_update
7. svn_checkout

Basically this is how a service like Springloops is built. I hope, you’ve enjoyed this article as much as I did to write it, heh heh.

Categories: OpenSource · PHP · pecl · subversion · ubuntu
Tagged: , , , ,

Installing PECL subversion extension for PHP in Ubuntu 7.10

April 21, 2008 · 4 Comments

I was trying to interact with my subversion repositories using PHP yesterday and I knew that PECL has a extension named “SVN” for PHP users. So I tried to install in in my machine by when I tried to install it with the following command it always failed saying “unable to locate svn_client.h

sudo pecl install -f svn

Then I googled for some time to find out which package actually and I found that it is a part of libsvn which I didnt install in my machine which has ubuntu 7.10. so I found the appropriate package using the following command and it obviously locate the appropriate lib to install, which is “libsvn-dev

sudo apt-cache search libsvn

After that I just installed it via the standard procedure “sudo apt-get install libsvn-dev” and then tried to install the pecl extension once again. And whoa!, it works.

Here’s a small snippet to find out the difference of a single file from the repository under two separate revisions. in the following example svn_diff function returns two streams. one of them contains the difference and another one contains the error block.


<?php
list($diff,$error) = svn_diff(”http://orchidframework.googlecode.com/svn/trunk/app/config/configs.php”,
52,
“http://orchidframework.googlecode.com/svn/trunk/app/config/configs.php”,
61);
echo “<pre>”;
fpassthru($diff);
?>

Easy, huh?

Categories: PHP · pecl · subversion

whoops! naked_women are being part of php frameworks - LOL!

April 12, 2008 · 15 Comments

i don’t know if anyone before me already noticed this funny and cheap trick that the developers of kohana framework (which is a fork of codeigniter) have integrated with their distribution. while surfing the code base, i’ve found a file name “Naked_Woman.php” as part of kohana, located under system/libraries.

The content of this file is funny as well.


<?php defined('SYSPATH') or die('No direct script access.');
/**
 * Merry Christmas!
 *
 * I wonder if this will be picked up by Google and cause a massive number of
 * hits to just this file....
 *
 * @author     Kohana Team
 * @copyright  (c) 2007 Kohana Team
 * @license    http://kohanaphp.com/license.html
 */
class Naked_Woman_Core {
	public function __construct()
	{
		throw new Kohana_User_Exception
		(
			'cash::generate(), plz?',
			'There are no naked women or cash generators in Kohana. Sorry! ;)'
		);
	}
} // End Naked Woman

didn’t know naked womens are so influencing to even php frameworks! - lol

Categories: PHP
Tagged: