The last regularly scheduled meeting of the DST4L class on November 26th, 2013 was devoted to course feedback (the originally scheduled topic “Putting it all Together: On the Web, PDF, Reporting” was changed to account for a short holiday week and the hackathon planned in January). Chris Erdmann opened the session by reiterating the original… Read More
“(11/17/2013) Guided once more by the wise and powerful Dr. Lynn Cherny, MD (Master of Data), we first attempted to discourage cardiac fibrillation by applying Topic Modeling to Grimm’s Fairy Tales, and we then performed an exploratory Gephi Procedure on the resulting corpus. Both efforts were considered successful and the Gephi Procedure revealed that data… Read More
On November 18th, Gabriel Florit from the Boston Globe talked about data analysis and exploration and presented a wide variety of his interactive visualizations to an attentive audience at the CfA Phillips Auditorium. Gabriel’s Natural Language Toolkit (NLTK) projects included text analysis of words used by Mitt Romney and Barak Obama during the Presidential campaign. NLTK… Read More
The DST4L class of November 4 featured the return of instructor Lyn Cherny, whom we last saw at the end of August when she taught a class on data manipulation and graphing in Excel. Lynn started her lecture by quoting a reporter she was talking to earlier this year, who said, “Data visuals are the… Read More
In our 9th week of #DST4L, Rahul Dave built on the previous week’s crash course in statistics with a deep dive into machine learning, classifiers, and information retrieval. First, this saw us working through further applications of scikit-learn, using a common pattern for classification: Create a classifier Fit the classifier with training values Run classifier.predict… Read More
Our #DST4L class on October 22nd was a whirlwind tour of probability, statistics, and machine learning in 3 hours. Before diving into that, I want to make sure you all know that the Python programming language was named after Monty Python. You may have figured this out if you’ve clicked on a Google result and… Read More
In this past week’s class on Tuesday, October 15th, we went back to our dataset of olive oils and their constituent fatty acids to learn some more about Pandas, NumPy, and MatPlotLib. As an aside, this course is doing strange things to my vocabulary. It takes me a bit to remember that pandas and pythons… Read More
Today we looked at exploratory data analysis and pandas using matplotlib. After hanging out with Tom Morris for the past five weeks, we switched it up and had a lecture from Rahul Dave. Rahul is a computational scientist at Harvard’s Astrophysics Data System. It had been a while since we’d played with Python, and there… Read More
On October 1, we had our fifth and final session with our able instructor, Tom Morris, for this section of the DST4L course. Tom provided us with an overview of APIs that was succinct but detailed. We learned how web APIs are used to tease data from web sites. For those who are new to… Read More
On October 4th, we were lucky to be joined by James Turk from the Sunlight Foundation, “a nonprofit, nonpartisan organization that uses the power of the Internet to catalyze greater government openness and transparency, and provides new tools and resources for media and citizens, alike. They are committed to improving access to government information by… Read More
This week we continued on our Data Scientist journey by exploring web scraping basics. Our intrepid leader was again Tom Morris, who demonstrated two ways, out of many, to scrape web pages. While there are plenty of tools online to accomplish this task, the focus of the class was on OpenRefine and Python. Before starting… Read More
Training on OpenRefine continues with instruction from expert Tom Morris. Over two weeks we have learned how to import different kinds of file formats, and explored the tools and techniques for reconciling names and other entries in our data. Reconciling is the process of matching elements in the data set to remote linked data sources.… Read More
OpenRefine has already proved a valuable tool in the workplace. Days after Tom’s first session I produced a set of CSV files with usage data from Harvard’s library reporting system. I wanted to upload and share the data in a Google spreadsheet, but first had to modify the date information to enable sorting and filtering.… Read More
On a rainy Thursday morning, Tom Morris (@tfmorris on Twitter) gave the class an introduction to OpenRefine. As is often the case with learning a new technology and using it on a variety of platforms, there were some mild growing pains. We spent the first twenty minutes of class setting up the software. While it… Read More
Lynn Cherny (@arnicas on Twitter) shared her Excel expertise with us at Round 2 of Data Scientist Training for Librarians (#DST4L) on August 28th. It was just 4 days after our 2-day Software Carpentry Bootcamp where we were drilled on Unix, Python, and Git skills. The Windows version of Excel is very different than in years past. … Read More
