Getting Started with Colorado Open Data

knitr::opts_chunk$set(
  collapse = TRUE,
  comment = "#>",
  warning = FALSE,
  message = FALSE
)

library(coOpenData)
library(dplyr)
library(ggplot2)

Introduction

Welcome to the coOpenData package, an R package designed to provide convenient access to the Colorado Open Data Portal.

The package provides a streamlined interface for discovering and downloading datasets from Colorado Open Data. It helps bridge the gap between raw Socrata API endpoints and tidy data analysis in R.

The package provides three primary functions:

co_list_datasets() for browsing available datasets
co_pull_dataset() for downloading datasets using a catalog key or Socrata UID
co_any_dataset() for downloading data directly from a Socrata JSON endpoint

Listing Available Datasets

The first step in a typical workflow is to use co_list_datasets() to retrieve the live Colorado Open Data catalog.

catalog <- co_list_datasets()

catalog

The returned catalog includes information about the datasets available through the portal. Two especially important columns are:

key, a human-readable dataset identifier generated from the dataset name
u_id, the official Socrata dataset identifier

You can search the catalog for datasets containing a keyword.

catalog |>
  filter(grepl("school", name, ignore.case = TRUE)) |>
  select(key, u_id, name)

Replace KEYWORD with a useful search term related to the example dataset selected for the package.

Pulling a Dataset

The primary way to download data is with co_pull_dataset().

A dataset can be requested using either its human-readable catalog key or its official Socrata UID.

Pulling by UID

example_data_u_id <- co_pull_dataset(
  dataset = "e4mf-ggwf",
  limit = 5
)

example_data_u_id

Pulling by Key

example_data_key <- co_pull_dataset(
  dataset = "denver_public_schools",
  limit = 5
)

example_data_key

Both calls should return data from the same dataset.

Keys and UIDs

Dataset keys are easier to read, while Socrata UIDs are more stable.

For reproducible research and long-term workflows, using the official Socrata UID is generally recommended.

Filtering Data

The filters argument can be used for simple exact-match filtering.

filtered_data <- co_pull_dataset(
  dataset = "e4mf-ggwf",
  limit = 25,
  filters = list(
    grade_levels = "ECE-5"
  )
)

filtered_data

You can confirm that the filter worked by inspecting the unique values in the selected field.

filtered_data |>
  distinct(grade_levels)

Multiple values can also be supplied.

filtered_multiple <- co_pull_dataset(
  dataset = "e4mf-ggwf",
  limit = 50,
  filters = list(
    grade_levels = c("ECE-5", "K-5")
  )
)

filtered_multiple

Multiple fields can be combined within the same filter list.

filtered_combination <- co_pull_dataset(
  dataset = "e4mf-ggwf",
  limit = 50,
  filters = list(
    grade_levels = "ECE-5",
    classification = "Charter"
  )
)

filtered_combination

Filtering by Date

If the example dataset contains a date or datetime field, records can be filtered using from, to, and date_field.

date_filtered_data <- co_pull_dataset(
  dataset = "e4mf-ggwf",
  from = "2023-01-01",
  to = "2024-01-01",
  date_field = "last_verified",
  limit = 100
)

date_filtered_data

The from date is inclusive, while the to date is exclusive.

A single day can also be requested using the date argument.

single_day_data <- co_pull_dataset(
  dataset = "e4mf-ggwf",
  date = "2023-09-06",
  date_field = "last_verified",
  limit = 100
)

single_day_data

Pulling Data from Any Socrata Endpoint

The preferred workflow is to use co_list_datasets() together with co_pull_dataset().

However, when a dataset is not available in the package catalog, co_any_dataset() can download data directly from a Socrata JSON endpoint.

Colorado Open Data endpoints typically follow this structure:

https://data.colorado.gov/resource/<dataset_u_id>.json

For example:

https://data.colorado.gov/resource/e4mf-ggwf.json

The endpoint can then be supplied directly to co_any_dataset().

endpoint_data <- co_any_dataset(
  json_link = "https://data.colorado.gov/resource/e4mf-ggwf.json",
  limit = 5
)

endpoint_data

Which function should you use?

Use co_pull_dataset() when the dataset is available through co_list_datasets().

Use co_any_dataset() when you already have a valid Socrata JSON endpoint or when the dataset is not included in the package catalog.

Example Analysis

Once the data have been downloaded, they can be analyzed using standard R tools.

The following example counts the number of records in a categorical field.

category_summary <- co_pull_dataset(
  dataset = "e4mf-ggwf",
  limit = 500
) |>
  filter(!is.na(grade_levels)) |>
  count(grade_levels, sort = TRUE)

category_summary

The results can then be visualized.

category_summary |>
  slice_head(n = 10) |>
  ggplot(
    aes(
      x = n,
      y = reorder(grade_levels, n)
    )
  ) +
  geom_col() +
  theme_minimal() +
  labs(
    title = "Most Frequent Categories",
    x = "Number of Records",
    y = "Category"
  )

This example demonstrates the complete workflow from discovering a dataset to downloading, filtering, summarizing, and visualizing it.

Summary

The coOpenData package provides a consistent interface for working with data from the Colorado Open Data Portal.

In this vignette, you learned how to:

browse available datasets using co_list_datasets()
download datasets by key or UID using co_pull_dataset()
filter data using fields, values, and dates
access a Socrata JSON endpoint using co_any_dataset()
perform a simple analysis and visualization

These functions allow users to focus on analysis rather than manually constructing API requests.

How to Cite

If you use this package for research or educational purposes, cite it using the package citation returned by:

citation("coOpenData")

Shelby Lyn Gomes