OpenElections logo

Earlier this week, we sent out a tweet as sort of a fishing expedition: "Political scientists: if you've collected precinct-level election results in your work & are willing to share, get in touch!"

kankakee-2000-resultsMike Sances did. An assistant professor of political science at the University of Memphis, Sances let us know that as part of his dissertation research he had obtained precinct-level results files for many counties in Illinois. In some cases, the files went back more than a decade. Even better, he was willing to share them with us.

Thanks to Sances, and to the National Science Foundation, which funded his research, we've begun posting the files - many PDF documents but some HTML and text files - to our Github repository for Illinois sources.

We won't have every county, nor every election, but this generous contribution to the project is exactly the sort of thing we were hoping to see when we asked. While OpenElections will be processing results data for state and federal races, we're also posting these files to allow people to find local race results as well - and to avoid requesting the same materials that Sances did for his research. These are public documents; they should be available to the public.

And we're renewing our call to political scientists: if you have precinct-level election results, either in original format or that you've digitized, we'd love to hear from you about making them a part of OpenElections.

 

 

Covering Oregon

31 Jan 2016

We've spent a lot of time in Oregon. Well, not in Oregon, physically, but searching for, obtaining and parsing election results from the state. County-level data is easy to find: the Secretary of State's office provides electronic PDFs of results for elections dating back to 2004. Thanks to Tabula, those were easy to convert so that we could load them into our database.

map

Source: http://www.oregon.gov/DHS/CHILDREN/FOSTERPARENT/PublishingImages/map.png

But precinct-level data was, as we've said before, a whole other story. We've written about specific aspects of this process, but here is the complete story about how we published what is, to the best of our knowledge, the first freely-available set of statewide precinct-level election results for 2012 and 2014 elections in Oregon.

We began gathering Oregon data in early May 2015, starting with the county-level data and then moving onto individual counties. We asked about other sources of precinct-level data, but never found a freely available one. The state does not maintain precinct-level results data. When we turned to the counties, we found a few that provided machine-readable results (mostly in PDF format), including the largest counties (Multnomah, Clackamas and Marion among them). Parsing those results was by no means simple, but the files were consistent enough to allow us to grab precinct results.

But for the bulk of Oregon's 36 counties, here was our process:

  1. Email county clerk's office asking about the availability of results. Sometimes we called.
  2. Request results for 2000-2014, if available. If not, request as much as they had.
  3. For some, pay a fee for the results, ranging from $37.50 (Union County) to $222.75 (Tillamook County, just for 2010-2014).
  4. Get the results, typically PDF image files that would require OCR to convert into machine-readable data. In some cases, we were sent paper via the postal service.
  5. Create election and county-specific files.

We had several volunteers (some listed here) help us with the conversion process, which helped us get the data converted faster. All told, we spent more than $1,000 on Oregon precinct-level results, and for that we got a wide range of files covering most of the elections we sought but not all.

We built a matrix showing the county coverage that we have, something that we'll be doing for other states and putting on openelections.net as well. In many counties, getting pre-2008 results was either impossible or cost-prohibitive. In one case, Crook County, the precinct results from the 2010 general and 2012 primary elections are no longer available. To repeat: no government entity has precinct-level results for Crook County for those two elections. They are missing from the public record.

Almost every county clerk we spoke or emailed with was friendly and wanted to help us. Many offered results for free online, and we happily downloaded those files. Others had results but have not posted them. Many of them appear to use the same software to produce election results reports but continue to print out those electronic reports and scan them, reducing legibility.

Oregon is, sadly, not an isolated case. There are other states where obtaining precinct-level results is a painstaking effort, even when counties and states produce electronic version of political maps and voter files. Election results, in too many cases, are a weak link in our civic infrastructure.

So we'll be back in touch with our Oregon clerks at the end of this year or early next year, asking for more election results. We'll know what to expect this time, and it should be a less-expensive and shorter process. But on both counts, the progress isn't enough.

 

As we wrap up OpenElections' work in 2015, we'd like to give you an update on how we've spent not only our time but also the money that we've gotten, particularly from the John S. and James L. Knight Foundation’s Knight News Challenge. Most of that money that we've spent since mid-2013 has gone to salaries for our project manager and a single developer. Neither of the project's co-founders has been paid for working on OpenElections, and we've tried to keep our operations pretty lean.

While our initial grant funding from Knight is nearly exhausted, we've made good progress and will keep going. In the past few months we've added a few more states (Louisiana, Missouri and Virginia) and we have volunteers working on Wisconsin, Georgia and Oregon, among others. We've revised our volunteer documentation to make it easier to understand what we're doing and how you can help.

In most states, getting county-level data isn't too much of a problem; that data is usually available online, if not always in native electronic formats. County-level data is usually freely available as well, but we've always wanted to develop a resource that can offer precinct-level results where they are available. Here's why: while counties can be homogenous, precincts are even more distinct and smaller political units and lend themselves to more sophisticated analysis. Candidates and their campaigns care about precinct results. Journalists and researchers should do the same.

Some states make precinct-level data available for free, which is a great service to the public. They include Louisiana, Maryland, Wyoming, West Virginia, Virginia, North Carolina and Florida, among others. Some states, like Pennsylvania, Colorado and Utah, charge a nominal fee for precinct results. But for other states, precinct results are only available county-by-county, and that takes both time and money. We've written about Oregon in the past, and we'd like to offer it as an example of the price of precinct results.

The bad news is that it's not uniform, even within a state. In Oregon we've spent more than $1,000 to obtain precinct-level results covering elections 2000-2014, although in many cases we don't have all of those years. Some counties literally don't have results to give us before 2010. Crook County was unable to find precinct results for the 2010 general and 2012 primary elections, while a number of counties don't have elections from before 2008. In other cases, price was a factor: we'll only have precinct results for 2010-2014 from Tillamook County because the clerk there charged us $222.75 for results for those years. Lake County charges $50 an hour for pulling the results files and another $0.25 a page for copying them. We've yet to receive those results, so we don't know what the final cost for Lake will be.

The good news is that when we do request election results that aren't freely available online, we're posting them on our Github site in state-specific repositories. That way other organizations or individuals won't have to repeat our processes and/or pay for results that we've already gotten. We want you to use what we've gathered, whether that's CSV files or original PDFs. That's our holiday gift to you. We'll be back at it in 2016, when there are more elections coming up, we hear.

In some states, getting election results data is pretty easy (looking at you, Maryland, Wyoming and Florida, to name three). In others, it's a matter of going county-by-county, as we've previously written. But in most of these cases, we've been dealing with results files that are available online.

But what about cases where the results aren't on the Internet?

You might think that's far-fetched, but it's not. As we've been working through Oregon, for example, we've come across a handful of counties that do not post precinct-level election results on county websites. For these counties, we've requested that data be sent to us, and we've gotten PDFs and spreadsheets and even paper in response.

Currently we have results from 10 Oregon counties that weren't previously available online. So we're publishing those, too, even before we convert them into data for our purposes. Here, for example, is the repository containing Oregon county files we've obtained (including those from Tillamook County, which we've written about). Posting those PDF files not only helps us build our collection, but it enables volunteers like Pete Huang to convert them into data, as he did for Harney, Wheeler and Morrow counties.

Oregon's not alone, we're finding. In states like Connecticut, which has few counties and towns and cities can be more meaningful political jurisdictions, we've added 2014 results files sent to us by volunteers. We've also started a collection for Kansas, which has yet to publish precinct-level data for any statewide races last year, despite doing so in earlier years. In both cases, we've gotten the files thanks to John Mifflin, a Washington state resident who has been collecting election results from counties and cities for a number of years and contributes data to the folks who work on Daily Kos Elections. John's work has made it possible for us to fill in gaps in our own efforts and we've also tried to help convert PDF and other image files into formats that he can use for his purposes.

While our main focus is on collecting and publishing county-level results, we've seen the need to make election results available in whatever form we can find them. If you're interested in helping out, we've got some "easy tasks" and we're always interested in hearing from people who have election results to share, too.

IMG_3129

A few weeks ago, we began requesting precinct-level election results from counties in Oregon. The Secretary of State maintains county-level results, typically in electronic PDFs, so to get down to precinct level we need to ask county clerks across the state. Many of them post precinct results on their sites, but some don't, so we emailed a few to ask for results from 2000-2014. In doing so, we were prepared to pay reasonable fees for them, as the Oregon Revised Statutes permit.

Local officials were quick to get back to us in every case, and their responses were straightforward. Here's an example, from Art Harvey, the Josephine County clerk and recorder:

The reports you are interested in are available in PDF format.

The cost would be $10.00 per election.

Other counties charged fees ranging from $25 (Umatilla County) to $45 (Wasco County) to $86.50 (Linn County, which sent us paper print-outs of election results that we'll be scanning). And then there's Tillamook County, where Tassi O'Neil, the county clerk there, has set a price of $664 for PDF copies of precinct-level results for elections from 2000-2014.

We wondered how that price was calculated, so we asked. Ms. O'Neil responded:

The fee for each election is $3.75 locate fee and then .25 cents per page.  That is the fee if it is a paper copy or if we send it in a PDF.  That is the charge that the Oregon Revised Statues say that we can/or should charge.

That is true, but there are two points here: One is that members of the public are being charged for pages of an electronic document. There are no paper copies involved here. The other is that the Oregon Revised Statutes also say this:

The custodian of any public record may furnish copies without charge or at a substantially reduced fee if the custodian determines that the waiver or reduction of fees is in the public interest because making the record available primarily benefits the general public.

As OpenElections is a non-profit effort dedicated to publishing machine-readable election results that can be freely used by anyone, we're pretty sure that our project primarily benefits the general public. We've asked for such a waiver or reduction of the $664 and are awaiting a reply. Oregon law also permits us to appeal a denial of a fee waiver or reduction to the Attorney General, and we will be pursuing that option should it become necessary.

In the meantime, we've been converting Oregon PDF results to CSVs and will continue to do so. There are plenty of ways for you to contribute to that effort, and we welcome any suggestions or advice on our dealings with Oregon officials.

2016

2015

2014

2013

2012