tweepy examples

Ulf Hamster 3 min.
python tweepy Twitter API

Load Packages

#!pip install tweepy
import tweepy
import pprint
pp = pprint.PrettyPrinter(indent=4)

Apply for a Token

You need to have an Twitter Developer account to use tweepy. Go the Twitter Developer dashboard, fill the forms, and wait some hours. Twitter sent an email with further questions as well. Answer these too, and wait for the approval email.

OAuth2 Authentification

# read credentionals from somewhere (Nope. I regenerated the Consumer Key after I was finished with this notebook)
consumer_key = "BhK3stez6xZoabztuOOn0v36g"
consumer_secret = "vkAYrdopSRnSg7JqtB40mTc5KgJw8hVZAHnZWMUYhRuIqTiLtc"
auth = tweepy.AppAuthHandler(consumer_key, consumer_secret)

Search some stuff

api = tweepy.API(auth_handler=auth, wait_on_rate_limit=True)
print(api)
<tweepy.api.API object at 0x1072b9978>
for tweet in tweepy.Cursor(api.search, q='machine learning overfitting').items(5):
    #pp.pprint(tweet._json)
    print(tweet.created_at, tweet.text)
2019-10-31 01:43:55 Koehn: So what's the outlook?

Open Questions
- why are neural models overfitting?
- can we detect hallucinations?… https://t.co/1eEzPJVvu7
2019-10-30 20:27:19 a funny thing is that the most successful strategies for machine learning involve slowing things down to prevent ov… https://t.co/XawdXOS6kG
2019-10-30 19:37:27 Reconciling modern machine-learning practice and the classical bias–variance trade-off
https://t.co/m3tMdclDbL
What… https://t.co/R0JbDbKqGG
2019-10-30 17:01:57 Overfitting and Regularization in Machine Learning https://t.co/nvTEds8Je2
2019-10-28 15:26:39 overfittingしてからvarianceを下げていくのがbagging
underfittingしてからbiasを下げていくのがboosting

https://t.co/7WVVE3HXLU

わかりやすい。

Where are the followers from?

user = api.get_user('bubblride')
#pp.pprint(user._json)
print(user.id_str)
899718305514217474

The results are limited to 20 rows

cnt_unknown = 0
places = []
for u in user.followers(count=200):
    #pp.pprint(u._json)
    if u.location:
        places.append(u.location)
    else:
        cnt_unknown += 1
print(cnt_unknown)
sorted(places)
3





['Bonn, Nordrhein-Westfalen',
 'Buenos Aires, Argentina',
 'Des Moines, IA',
 'Ecotopia',
 'GLO, HNL, SEA, and growing',
 'Ipswich, England',
 'Melbourne, Australia',
 'Melbourne, Australia',
 'Nagpur, India',
 'Neenah, Wisconsin 54956',
 'New York, NY',
 'New York, NY',
 'New York, USA',
 'New York, USA',
 'Ocean Beach, San Diego',
 'Palo Alto, CA',
 'Paris, France',
 'Philadelphia, PA',
 'Portland, OR',
 'Portland, OR',
 'San Diego, CA',
 'San Francisco, CA',
 'Seattle Washington',
 'Singapore',
 'Singapore',
 'Sion, Switzerland',
 'United States',
 'china,xian']

The Websites of the accounts I'm following

user_id="899718305514217474"
%%time
websites = []
cnt_unknown = 0
for u in tweepy.Cursor(api.friends, id=user_id).items():
    #pp.pprint(u._json)
    try: 
        url = u.entities['url']['urls'][0]['expanded_url']
        websites.append(url)
    except:
        cnt_unknown += 1
CPU times: user 448 ms, sys: 53 ms, total: 501 ms
Wall time: 8.27 s
print(cnt_unknown)
sorted(websites)
17





['http://AlgoTransparency.org',
 'http://ChrisAlbon.com',
 'http://FakerFact.org',
 'http://GoCatalant.com',
 'http://about.me/jramphis',
 'http://about.me/kylepolich',
 'http://ai.google',
 'http://aimasters.de/de/',
 'http://alex.combessie.com',
 'http://alexanderswhite.com',
 'http://amueller.github.io',
 'http://angel.co',
 'http://angelabassa.com/faq',
 'http://bayesball.blogspot.com',
 'http://becomingadatascientist.com',
 'http://biffud.com',
 'http://biopolymersforlife.org',
 'http://bit.ly/328TP1u',
 'http://bit.ly/AISuperpowers',
 'http://bit.ly/vrsamVR',
 'http://blog.echen.me',
 'http://blogs.csc.com/author/doing-data-science/',
 'http://brunaw.com/',
 'http://cecinestpasun.com/',
 'http://channels.theinnovationenterprise.com/',
 'http://civisanalytics.com',
 'http://cliffano.com',
 'http://cognitivescale.com',
 'http://constantineckner.com',
 'http://daroczig.github.io',
 'http://datadan.io',
 'http://datascience.salon',
 'http://datastori.es',
 'http://derwen.ai/paco',
 'http://emerj.com',
 'http://emerj.com/',
 'http://erikbern.com/',
 'http://evolutionmedicine.com',
 'http://factcheck.org',
 'http://faculty.washington.edu/ebender',
 'http://fcnationpod.com/',
 'http://fivethirtyeight.com/',
 'http://gitlost.net/',
 'http://hadley.nz',
 'http://hilaryparker.com',
 'http://hrvtraining.com',
 'http://jameshewitt.net',
 'http://jm-contreras.github.io/',
 'http://joelgrus.com',
 'http://kaggle.com',
 'http://kaggle.com/learn',
 'http://knowledger.rbind.io',
 'http://lineardigressions.com',
 'http://lineardigressions.com',
 'http://linkedin.com/in/suzanailic',
 'http://localmaxradio.com',
 'http://machine-ethics.net/',
 'http://marcobonzanini.com',
 'http://medium.com/@julsimon',
 'http://melbourne.measurecamp.org/',
 'http://nickstrayer.me',
 'http://nssdeviations.com/',
 'http://numer.ai',
 'http://numer.ai',
 'http://odsc.com',
 'http://oreillyaicon.com',
 'http://pbpython.com',
 'http://pytorch.org',
 'http://quantifiedself.com/',
 'http://rctatman.com/',
 'http://sebastianraschka.com',
 'http://sigmoidal.io',
 'http://simplystatistics.org',
 'http://spielverlagerung.com',
 'http://srome.github.io',
 'http://storybydata.com/',
 'http://strataconf.com',
 'http://talkthetalkpodcast.com',
 'http://theblasfrompas.blogspot.com/',
 'http://tryolabs.com',
 'http://turing.ac.uk',
 'http://twimlai.com',
 'http://unsupervisedthinkingpodcast.blogspot.com/',
 'http://varianceexplained.org',
 'http://www.ConvergenceScienceNetwork.org.au',
 'http://www.DominoDataLab.com',
 'http://www.TuringFinance.com',
 'http://www.acerta.ca',
 'http://www.aihardwaresummit.com',
 'http://www.aiworld.com',
 'http://www.aliresearch.com/en/index.html',
 'http://www.americansocceranalysis.com',
 'http://www.analyticsvidhya.com/',
 'http://www.andrewng.org',
 'http://www.appliedMLdays.org',
 'http://www.appnomic.com',
 'http://www.biostat.jhsph.edu/~rpeng/',
 'http://www.buergerschaffenwissen.de',
 'http://www.burrus.com',
 'http://www.coriniumintelligence.com',
 'http://www.crowdanalytix.com',
 'http://www.cs.bath.ac.uk/~jjb/',
 'http://www.cs.cmu.edu/~atalwalk/',
 'http://www.datascienceimposters.com',
 'http://www.deepmind.com',
 'http://www.doherty.edu.au',
 'http://www.drivendata.org',
 'http://www.ecohealthalliance.org',
 'http://www.enlitic.com',
 'http://www.figure-eight.com',
 'http://www.fivethirtyeight.com',
 'http://www.hannahishere.com',
 'http://www.healthtech.com',
 'http://www.huber.embl.de',
 'http://www.iem.uzh.ch',
 'http://www.jatana.ai',
 'http://www.jobsfornewdatascientists.com',
 'http://www.jtleek.com',
 'http://www.kyndi.com',
 'http://www.lifeonashirt.com',
 'http://www.lse.ac.uk/european-institute/people/academic-staff/Chris-Anderson',
 'http://www.meetup.com/Deep-Learning-London/',
 'http://www.nature.com/natmachintell',
 'http://www.nesta.org.uk',
 'http://www.nmrql.com',
 'http://www.oben.com',
 'http://www.ospreyinformatics.com',
 'http://www.paperspace.com',
 'http://www.predictiveanalyticsworld.de',
 'http://www.pyimagesearch.com',
 'http://www.r-bloggers.com',
 'http://www.randalolson.com',
 'http://www.research.ibm.com/',
 'http://www.researchsquare.com',
 'http://www.rstudio.com',
 'http://www.rstudio.com/',
 'http://www.seez.co',
 'http://www.selerityinc.com',
 'http://www.sitezeus.com',
 'http://www.snopes.com',
 'http://www.spacex.com',
 'http://www.startupvictoria.com.au',
 'http://www.superdatascience.com',
 'http://www.thepfsa.co.uk',
 'http://www.thoughtworks.com',
 'http://www.tomtomfest.com',
 'http://www.voicea.com',
 'http://www.youtube.com/sirajraval',
 'http://yihui.name',
 'http://youtube.com/philosophybehindcoding',
 'http://zapier.com',
 'https://about.me/ravimistry',
 'https://ai.facebook.com/blog',
 'https://ai.vub.ac.be',
 'https://aka.ms/yynzoe',
 'https://anttilip.net/',
 'https://bit.ly/2GLC0yq',
 'https://bit.ly/33eLuta',
 'https://caserta.com',
 'https://catherinehavasi.com',
 'https://changelog.com/practicalai',
 'https://chris-said.io',
 'https://chrisbenson.com',
 'https://cnvrg.io',
 'https://engineering.doximity.com',
 'https://fchollet.com/',
 'https://forwardleading.co.uk/summits',
 'https://fullfact.org',
 'https://gitcoin.co/mission',
 'https://ines.io',
 'https://jaumo.onelink.me/30298309/Twitter',
 'https://khakieconomics.github.io/',
 'https://medium.com/@radekosmulski',
 'https://openai.com',
 'https://plus.google.com/107322247125350038821',
 'https://poplindata.com',
 'https://popup.dominodatalab.com/',
 'https://pronoun.is/she',
 'https://sites.google.com/view/nancypadilla/home',
 'https://thingsthatchangethewayithink.blogspot.com/',
 'https://twiecki.io',
 'https://twimlai.com',
 'https://www.ai-expo.net/',
 'https://www.crossover.com/',
 'https://www.dataschool.io',
 'https://www.experfy.com',
 'https://www.facebook.com/artificialintelligencememes/',
 'https://www.kaggle.com/jobs',
 'https://www.kdnuggets.com/',
 'https://www.linkedin.com/in/mattmayo13',
 'https://www.linkedin.com/in/susanetlinger/',
 'https://www.lucymcgowan.com',
 'https://www.meetup.com/gdg-melbourne/',
 'https://www.microsoft.com/en-us/research/people/cmbishop/',
 'https://www.shinesolutions.com',
 'https://www.softwareone.com/en-us/',
 'https://www.superdatascience.com/']

Links