if you want to get all tech news update (or php news, did i forget to say python news or web 2/3/x news??) in one place, what could be better than a twitter account or a RSS feed reader. If you have a RSS feed aggregator, its plain and simple. But if you choose the other one (I mean Twitter) you have to be something more than a dumbass, as you have to create it by your own- lol!
so let me be something more than a “dumbass” today and show you how did i do it. i choose my all time best tool “php” for it. added some toppings with an external rss mashup builder and finally served the dish with twitter api. lets see what was the difficult part in it
goal:
you will provide a set of rss feed urls, then your application will create a mashed up feed with them (all in one place, sorted in ascending order by date), and publish the latest feed titles since last time update.
challenges
0. building a mashup with rss feed
1. keeping track of last time update and separate items which are new
3. post them to twitter
4. setting up cron to run it periodically
getting hand dirty
step 1: creating a rss feed mashup
so lets build a mashup of your favorite rss feed urls. you can make use of yahoo pipes for this or take help of any other web based mashup service. for this one, i experimented with the one available at xfruits.com. go there, open an account and add your feed urls. xfruit will give you a single feedurl which will deliver the mashed up content, sortedn in ascending order by date. so this part is done.
[update]
I tried with Xfruit but it doesn’t work well, most of the time it is down – and their update frequency is slow. so i build the mashup with yahoo pipes. here is the source
–
after making a mashup of techcrunch, phpdeveloper, insidefacebook, ajaxian and honeytechblog my final mashed up feed url (via yahoo pipes)is http://pipes.yahoo.com/pipes/pipe.run?_id=bDfi0Pww3hGhpiLVPm7D0g&_render=rss
step 2: coding your little franky, the frankenstein
well, this code makes use of a tinyurl API to shorten the url of each feed story, SimpleXML to parse the RSS feed and extract stories and story-links from it, a text file to keep track of last updated content and twitter API to post them to twitter. here they come one by one.
class.tinyurl.php
< ?php /** * shorten any url with the help of tinyurl API * */ class TinyUrl { static function getTinyUrl($url) { $tinyurl = file_get_contents("http://tinyurl.com/api-create.php?url={$url}"); return $tinyurl; } } ?>
class.cachemanager.php
< ?php /** * keeps track of updated items since last execute * */ class CacheManager { private $lastHash; function __construct() { $this->lastHash = file_get_contents("hash.txt"); } function getLashHash() { return $this->lastHash; } function setLastHash($hash) { $this->lastHash = $hash; file_put_contents("hash.txt",$hash); } } ?>
class.twitter.php
< ?php /** * update user's status in twitter * */ class Twitter { private $username; private $password; function __construct($username, $password) { $this->username = $username; $this->password = $password; } function publishMessage($message) { try{ $tweetUrl = "http://{$this->username}:{$this->password}@www.twitter.com/statuses/update.xml"; $curl = curl_init(); curl_setopt($curl, CURLOPT_URL, $tweetUrl); curl_setopt($curl, CURLOPT_CONNECTTIMEOUT, 2); curl_setopt($curl, CURLOPT_RETURNTRANSFER, 1); curl_setopt($curl, CURLOPT_POST, 1); curl_setopt($curl, CURLOPT_POSTFIELDS, "status={$message}"); $result = curl_exec($curl); //print_r($result); } catch(Exception $e) { //print_r($e); error_log($e->getMessage()); } } } ?>
now the most interesting part, to make use of all your objects and to cook them properly. here is the code of index.php
index.php
< ?php include_once("class.cachemanager.php"); include_once("class.tinyurl.php"); include_once("class.twitter.php"); $twitter = new Twitter("username","password"); $cm = new CacheManager(); $lastHash = $cm->getLashHash(); $firstHash= ""; $feedUrl = "http://pipes.yahoo.com/pipes/pipe.run?_id=bDfi0Pww3hGhpiLVPm7D0g&_render=rss"; $rss = new SimpleXMLElement($feedUrl,LIBXML_NOCDATA, true); $feedcount = count($rss->channel->item)-1; $items = array(); /** * process elements to get a list of items to post in twitter * also set a check point so that items are not double posted */ for($i=0;$i< =$feedcount;$i++) { $title = $rss->channel->item[$i]->title; $link = $rss->channel->item[$i]->link; //echo "{$title}<br />{$link}<hr />"; if($i==0) $firstHash = md5($title); $hash = md5($title); if($lastHash!=$hash) { $shortlink = TinyUrl::getTinyUrl($link); $items[] = array("title"=>$title,"link"=>$shortlink); } else break; } /** * set the checkpoint */ $cm->setLastHash($firstHash); /** * post to twitter */ $items = array_reverse($items); foreach ($items as $item) { $message = "{$item['title']} - {$item['link']}"; $twitter->publishMessage($message); echo $message."<br />"; } echo "Whoa, Done Finally!"; ?>
so you are done coding it. now set it as a cron job (you can use http://www.setcronjob.com if you are looking for a free one) from your cpanel. you are done.
You can check my twitter news updater at http://twitter.com/techupdater
Just eased my exploration with Twitter.. thanks 😛
http://twitter.com/nine_L
Boss… ghuman kemne ato buddhi loiya?
BIGGG THANKS for this tricks 😀
sweet code, with a little mods can be used with plurk or FB 😉
Thanks for the excellent job. it will help me a lot….
Cool tips. Apnar matha niya gobeshona kora dorkar.
Great Work… as always. Thanks Hasin bhai! Just recently I started to use twitterfox. Now I can take full advantage of it by adding all my RSS feeds to twitter with this code. Whole world will be waiting for me on my firefox 🙂
🙂
Thanks a lot . 😀 i was looking for this !
this is similar to this: http://richardxthripp.thripp.com/tweet-this
Cool!
@TaeWoo – that is really similar! thanks for the link – i didnt know abt that one.
@TaeWoo : I have been using Tweet this but that has basic difference to this one.
1. That’s WP plugin.
2. Doesn’t use cronjob.
3. Dont automatically post to tweeter.
4. That is just an aid to the Blog visitors to be able to Tweet a liked post easily.
5. This one aggregates RSS feeds and automatically posts to ONE single Tweeter Account.
🙂
nice trick! liked it. thanks for sharing. 🙂
wow!
Thanks for the post.
What about the restriction of the number of feeds that are sent? You do not want to bombard people with too many in one time.
it’s definitely a great job, it’s really a new idea! to me.
I am wondering if everyone start using ur code, we may be flooded by feed update, instead I am thinking to follow your http://twitter.com/techupdater
thanks,
Excuse my ignorance but do I update password in index.php or class.twitter.php or both
hi azzam, please update only in index.php 🙂
i faced a problem after running the code….
the problem shows me that…Maximum execution time of 30 seconds exceeded …please suggest me
errr… twitterfeed anyone? http://twitterfeed.com
Nice work 🙂 You saved lots of time 😛
Thanks.
I am going with twitterfeed.com, it makes it all that easier
@Azzam – u r most welcome 🙂
i use google reader actually. its more handy to me.
dont forget take a try. however thanks for sharing your idea
I must say that it is handy since it delivers the feeds from the web and not from any other tool i.e ‘twitterfeed’, makes it look organic and naturally.
Too many that look as if they are from RSS can be off putting.
I suggest Hasin that a) Allow feeds to be sent on a timed basis 30 min, 1 hr, 4 hour, 24 hour, etc.
b) they number of post submitted per tweet i.e. 5, 10, 15, 20, etc.
that would be awesome
@kowser, I also use google reader with the greasemonkey/userscript for twitter. Works a treat with tinyurl. The fact that you send only the ones you have read manually ensures that you are only sending what you have read and confirmed what is good for your readers.
awesome post. thanks hasin vai for this nice tips and tricks. 🙂
Very useful. Thanks for the nice article.
excellent OOP example for feed!
oneker darooon kaje ashbe!
Came to your site after longgg time, looks like missed a lot of fun 🙂
I have been looking for the exact thing you posted(RSS Feed)! Thanks a lot Hasin.
Am planning to sit for the Zend certifcate soon, any suggestion on that? Or do you already have a post for that I can look into?
Once again! Thanks for all EXCELLENT posts!
Please excuse my ingorance I am new to this. I have edited the index.php line 5 $twitter with my username and password. I’ve uploaded the files above to a dir on my server. When I run http://www.mydomain.com/bot/index.php the page shows me the text of my index.php file incuding my password. Please help. I have set this up in dreamhost cronjob as well but I nothing seems to work. Thanks in advance for you help.
I have the cronjob executing the index.php file that I uploaded to a directory with the rest of the files but now my email notifier tells me:
sh: 20: command not found
Also if i browse to the index.php file in my browser it’s still showing me the text of the .php file in my browser.
@Pete: In all the PHP files listed above, There is a space between < and ?php, so remove the space and Try.
The code is working fine as along as there is no special characters in the item title.
If the item title has any spl char, only the partial text of the item title ( until it hit the spl char) is updating to Twitter.
Can you please look into this. Thanks.
i mean characters like & < > "
added a new line
$message = urlencode($message);
before
$twitter->publishMessage($message);
working fine for the above mentioned special character issues.
Thanks
Khub e kajer jinish!! Great idea and implementation 🙂
Try FeedMingle http://feedmingle.com instead of yahoo pipes or xfruits. It is much easier and performs great.
Amazing Script! I had one question:
1. It currently says “posted from web” when posted to twitter – would you know how to change the text of that like twitterfeed or ping.fm do?
Very cool tutorial, I’m definitely going to take this and try to create a few little implementations. Very handy stuff indeed.
Great Post! Love your blog! Btw I stumble onto this site that provides a Rss tool that converts any partial rss to full rss! Just wanna share with you guys: Full rss Have fun with it!
baidu