September 2, 2014 - 29 comments

Exploring the Twitter API with Meteor.js

Exploring the Twitter API and Meteor.js

The Twitter API is thoughtfully designed, and the data set that it provides access to is incredibly rich.

I wanted a super quick, no-fuss way to retrieve, process, and display tweet data on the web, and have been obsessed with Meteor.js ever since I started using it on our pet project Songnotes. It's easier to set up an app and run a dev server than any other method I've encountered, and it keeps everything in JavaScript, meaning that I can display the data directly on the web with no intermediate hassle. Plus it just feels so cutting edge. Since I didn't find a straightforward tutorial on how to do this elsewhere on the web, I've outlined the steps I came to below.

Step 1: Create your Meteor app

If you've never tried Meteor before, the simplicity of installing it and creating your first app will blow your mind. You can follow the quick start, or try the tutorial on creating an example app first if you prefer.

Step 2: Obtain a Twitter access token

In order to access the API, Twitter requires an access token. There are a number of ways to get one depending on what you expect to be doing; I opted for the standard single-user developer token because it was easy to obtain and would give me all the access I'd need for the kind of data mining I want to do.

Step 3: Install a Twitter API client library

There are several different wrapper libraries to simplify access to the Twitter API  from Node (Meteor runs on a Node server, FYI) and twit seemed to be the most concise and straightforward to me, with a fairly large user base and frequent updates.

First enable NPM modules in your Meteor app

Running pure Node packages in a Meteor app requires first installing support for non-Meteor npm modules within your Meteor app:

meteor add meteorhacks:npm

Then add the specific package you need

Create a file called packages.json in the root of your app and add the package name and version number (1.1.19 at the time of this writing):

{
    "twit": "1.1.19"
}

And you're ready to go! Let's write some code.

Step 4: Do a test query

Open up myapp.js (or whatever the name of the main app file is that you created in Step 1), and edit the following section:

if (Meteor.isServer) {
  Meteor.startup(function () {
    // code to run on server at startup

  });
}

to look like this:

if (Meteor.isServer) {
  Meteor.startup(function () {
    // code to run on server at startup

    var Twit = Meteor.npmRequire('twit');

    var T = new Twit({
        consumer_key:         '...', // API key
        consumer_secret:      '...', // API secret
        access_token:         '...', 
        access_token_secret:  '...'
    });

    //  search twitter for all tweets containing the word 'banana'
    //  since Nov. 11, 2011
    T.get('search/tweets',
        {
            q: 'banana since:2011-11-11',
            count: 100
        },
        function(err, data, response) {
            console.log(data);
        }
    );

  });
}

filling in the '...' bits with the keys you obtained in Step 2.

It's important that sensitive information like certain API keys only be used within a Meteor.isServer block (or in a server-only file) in Meteor, otherwise your keys will be visible for all the world to see.

Now run your app by typing meteor on the command line from your app directory. If all went well, it should spit out a ton of json-structured content along the lines of

 { statuses: 
    [ { metadata: [Object],
        created_at: 'Mon Jun 23 04:06:56 +0000 2014',
        id: 480924945586069500,
        id_str: '480924945586069504',
        text: '@Nukes_Pringles banana',
        source: 'Twitter for iPhone',
        ...

Twitter is talking to us! Sweet. There are are more example queries to experiment with over at the twit github page, and in a future post I'll talk about what we might actually do with this data and how to display the parts we want on a web page.

Updated 1/27/2015 to use the latest version of Meteor, 1.0.3.1.

Published by: Paul Fernandez in Code

Comments

Justin Seiter
November 27, 2014 at 8:21 am

Great tut! Really looking forward to the second half of this as I’m guessing it will cover Meteor’s wrapAsync functionality.

I believe using wrapAsync should be my next step here as my Meteor.methods call is returning ‘undefinded’ — https://gist.github.com/justinseiter/57490323ce5e5411f6f0

    Paul Fernandez
    December 4, 2014 at 6:23 pm

    You might be right about needing _wrapAsync() if you want to fetch the data in advance in order to do some processing to it, after which you would store the results in a collection for later use. I was actually thinking of doing the next post client-side, though, and assigning the results to a Session() variable so that the view will update reactively once the data is done being fetched. This of course would limit the amount of data analysis you can do prior to redering the results. I’ll give it some thought; maybe I’ll do both techniques.

DigitalWheelie
December 2, 2014 at 5:27 pm

What version of Meteor did you use for this?
Wondering if this will work w/ Meteor 1.0.

Thanks!

    Paul Fernandez
    December 4, 2014 at 5:41 pm

    I wrote this post when it was at version 0.8.1.3, and it’s almost certain to need updating. I’ll be sure to update it prior to writing the next post.

gg
January 19, 2015 at 6:53 am

Would you able to update this yet? Thanks very much.

gareth dunne
January 27, 2015 at 12:03 pm

Can you please update this for Meteor 1.0 I can’t seem to get anything working and I’d really appreciate it.

    Paul Fernandez
    January 27, 2015 at 1:12 pm

    I’ve just updated it to Meteor version 1.0.3.1. Thanks for the reminder. 🙂

      Sander
      January 29, 2015 at 10:15 am

      The Meteor.require() method is deprecated, you might have missed that when updating. Meteor.npmRequire() should be used instead.

Jagan
February 17, 2015 at 9:37 pm

Great post. I followed the tutorial and was trying T.post at isClient.

How do I access the variable T (defined in server) here at the client side?
Is there a way I can use subscribe and publish to access T at client code?
I’m not using mongo.

Thanks,

    Paul Fernandez
    February 18, 2015 at 10:41 am

    I was going to do exactly what you’re saying you don’t want to do: store it to a Mongo collection and then publish it to the client. The reason is that doing a call to Twitter, waiting for the response, and then processing the data takes time and could make the application seem laggy to an end-user. However, you could also do a server-side function call as illustrated in this gist.

      Jagan
      February 21, 2015 at 1:46 pm

      Thanks Paul, I declared T, wrote the method on server side and did a meteor.call on client js. It worked.

      Now, I’m using twitter accounts package for user login and I do not find a way to dynamically get the access token of a logged in user. How can this be achieved? I tried Meteor.user() in console to see if access token/secret is present. No luck.

      Thanks again. Would love it if your next tutorial covers creating a buffer clone. 🙂

        Paul Fernandez
        February 24, 2015 at 8:25 am

        That’s great. I’d be interested to know the amount of data can be queried, and how much post-processing can be done without having the user feel that they are waiting too long for results.

        I don’t know about getting a user’s access token; the one used in this tutorial is for developers and is generated when the app is registered with Twitter, hence the need to keep it hidden server-side. My intent was to use my own token to scrape large amounts of general data from Twitter.

        abtx
        March 2, 2015 at 4:40 am

        Hi Jagan,
        I’m facing the same issue. Did you find a solution?

        Thanks

abtx
March 2, 2015 at 4:38 am

Hi, I get an issue when posting, Tweets are always being posted to my Twitter account where the Twitter app is created, no matter what other accounts I use to log in. What is the solution? Thanks

    abtx
    March 2, 2015 at 8:11 am

    I found a way to do this. Use Meteor.user().services.twitter.accessToken in methods, then pass it to your server side function

Grisham
May 8, 2015 at 3:36 pm

Please show us how to display select segments of this data on the UI

    Paul Fernandez
    May 8, 2015 at 6:43 pm

    My plan is to do that on the next post. If/when I can ever find time!

      Josie
      May 10, 2015 at 6:00 am

      Yes please, how do we move a certain value in these objects to a tag in our HTML file?

        GFargo
        July 10, 2015 at 7:58 am

        Josie, seems like you might be barking up the ‘Data Context’ tree. I recently read this terrific post by Sacha Greif.

        He goes into great detail about passing data between models and HTML templates. It seems like there are multiple ways to pass Data into the view so I’d be interested to know what worked best for you. Hope this Helps!

      GFargo
      July 10, 2015 at 7:50 am

      Terrific Tutorial, really looking forward to the follow up!

Elena
May 13, 2015 at 12:36 am

Hey Paul, how do we pass this data on to client side and show it on UI. Is there a tutorial for it? Thanks

Aaron
July 13, 2015 at 5:30 pm

Hey, thank you for writing this up; extremely helpful. Any chance you’ll be posting how to get this stored and displayed?

Again, great tutorial.

    Paul Fernandez
    July 13, 2015 at 7:42 pm

    Thanks. I’m working on the next post and hope to have it up very soon. In the meantime I have a simple working example of bringing the data to the front end on Github if you want to jump ahead.

Midhun
February 29, 2016 at 5:42 am

Hello..Thanks for the blog. I followed the procedure told by you, but got stuck.
I wrote the code in the server..but i want to return the result to client and display it. When consoling, i got the tweets on the server side but how to return this data to the client side which calls the method?

Kalice
March 2, 2016 at 7:28 am

Hi. I am using email for my user to sign/login. But after they’re login, I want to add their Twitter login credential. So they have 2 accounts in my application. How could I do this? Please help. Thanks

Leave a Reply