Blog Category: Code

Slurping Up Excel Data on the Quick: Python, Pandas, and Pickle by Ben Klaas · February 14, 2017
If you have very large tables of data imprisoned in a vendor-locked Excel jail, consider setting them free by caching worksheets using Python+Pandas+Pickle.
Read More ›

Towards a Sustainable Excel by Ben Klaas · February 3, 2017
Building Excel Macros With Python, part 3 of a series on reinventing our metadata management environment.
Read More ›

Automated Analysis of a Data Workflow - Part 2 by Jesse Erdmann · September 14, 2016
The conclusion of the story of how we created DCP Analytics - our in-house automated, web-based analysis tool using Pandas, Bokeh, Jupyter and Conda to help our researchers quickly find data anomalies and processing errors in our data production pipelines..
Read More ›

Automated Analysis of a Data Workflow - Part 1 by Jesse Erdmann · August 24, 2016
The story of how we created DCP Analytics - our in-house automated, web-based analysis tool using Pandas, Bokeh, Jupyter and Conda to help our researchers quickly find data anomalies and processing errors in our data production pipelines.
Read More ›

Excel VBA and Version Control by Jimm Domingo · May 19, 2016
The second post in the series about Team Unicorn Rainbows' work in the first round of MPC IT Shark Tank.
Read More ›

Improving Menu Creation in Excel with VBA by Jimm Domingo · February 25, 2016
In this series, we present some highlights from Team Unicorn Rainbows' work in the first round of MPC IT Shark Tank. This first post describes how we improved menu creation in Excel.
Read More ›

High Performance Analysis of Big Spatial Data by MPC IT · November 18, 2015
Our own HPC specialist Ankit Soni and the TerraPop team presented their published article at the IEEE Big Data 2015 conference in Santa Clara earlier this month.
Read More ›

Importing Fixed Length Data Using Ruby (Part Two) by Colin Davis · May 7, 2015
A follow-up to my post discussing my 'hflr' Ruby gem for reading hierarchical data in FLR format, today I'll demonstrate how to combine 'hflr' with a simple importer class to load a database with the data.
Read More ›

Fixed Length Record Data by Colin Davis · January 28, 2015
Dealing with fixed-length record (FLR) data is a reality for us at the MPC. Colin introduces readers to his Ruby Gem, HFLR, which makes processing hierarchical fixed-length record data a bit easier.
Read More ›

Ember for Rails Devs: Understanding How Ember Thinks by Jake Wellington · December 10, 2014
An introduction to Ember.js for devs who are used to thinking in Rails.
Read More ›

Keeping it Simple: Exploiting CSV and csvkit at the MPC by Ben Klaas · November 21, 2014
How we use csvkit to wrangle data around here.
Read More ›

Feeling Fuzzy: Name Matching at the MPC by Fran Fabrizio · October 10, 2014
The MPC's data has been cited thousands of times. In this article, we explore how we connected those citations with our user accounts using fuzzy name matching.
Read More ›

Data Duplication Detection by Jesse Erdmann · October 8, 2014