Book References Organizer

Building Blocks of a Personal Library & Notebook

Knowledge Organization · Lawyer Who Codes · Python · Markdown · YAML · Citation Style Language · Open Library API

A. Project Overview

I developed a Book References Organizer with two distinct spaces: (a) a Reading List, a lightweight repository designed to capture and track incoming resources (books, articles, and web pages) before they clutter the workspace; and (b) a Personal Library & Notebook, a long-term knowledge base where the user keeps personal notes alongside a collection of past and current readings. A separate JSON file is created for each space.

The Book References Organizer works in three main steps:

  1. (1) The program queries the Open Library API to fetch bibliographic data and saves these references in a local JSON Reading List.
  2. (2) The JSON data is filtered and restructured using Citation Style Language (CSL) as a reference data model, preparing it for import into the user’s Personal Library & Notebook.
  3. (3) sThe program converts the structured book data into YAML frontmatter and automatically writes it at the top of a new Markdown file.

The code is organized into five functional modules:

  1. I. Open Library Search Methods
  2. II. Reading List: Querying and Data Collection
  3. III. Personal Library & Notebook: Data Formatting
  4. IV. Markdown & YAML Frontmatter Generation
  5. V. Book Visualization & Index Lookup

The first module handles queries to the Open Library database. Modules II through IV contain the code for the three steps described above. Module V introduces two utility functions designed to be shared and reused across different areas of the program.

B. Installation

To run the code, the following third-party Python libraries must be installed:

The JSON and datetime modules are part of Python’s standard library and therefore do not require separate installation.

Since the program uses the Open Library API, it is important to comply with the applicable Usage Guidelines, in particular by sending requests with a User-Agent header identifying the application and providing contact information.

C. Why I Use Markdown and YAML

I have been using Markdown every day for quite some time, not only for taking notes and general writing, but also for storing different kinds of resources. These include course materials, excerpts from books, newspaper articles, and content found online.

Markdown allows me to focus on structuring my text and refining my line of reasoning without worrying about layout or presentation.

YAML (“YAML Ain’t Markup Language”) provides a human-readable way to represent structured data. Many modern Markdown editors, including Typora, Obsidian, and Zettlr, allow users to prepend a YAML frontmatter block to a document. This block can store structural metadata about the file, such as its author, modification date, or bibliographic references.

Because the frontmatter is clearly separated from the body of the text, a Python script can parse it independently from the rest of the Markdown file. Data can also be converted between YAML and JSON with relatively little processing.

Writing bibliographic data by hand in YAML is nevertheless tedious and error-prone, particularly because indentation is significant. YAML uses indentation to represent structure, so a misplaced space can cause errors that are difficult to spot. This project automates much of that process by retrieving bibliographic data from Open Library and progressively restructuring it.

D. Open Library API

1. Why I Chose Open Library

Several databases that catalog library collections provide APIs. After considering different options, including WorldCat, ISBNdb, and Crossref, I ultimately chose Open Library.

Open Library, an Internet Archive project, aims to create “one web page for every book ever published”. It provides open bibliographic data and a documented API, making it particularly suitable for a personal project based on book metadata.

For this project, I used requests to query the Open Library API and work with the JSON responses.

Open Library’s data schema also helped me understand how its records are structured.

2. Requests to Open Library

2.1. First Request Using an ISBN

The program searches for a book in Open Library using its ISBN. Before submitting a request, it verifies that the ISBN structure is valid. This does not guarantee that a book actually corresponds to the input, but it prevents malformed ISBNs from being submitted. Users may enter an ISBN-10 or ISBN-13, with or without hyphens.

Before 2007, ISBN-10 was the standard format. ISBN-13 was introduced to expand numbering capacity and align ISBNs with the EAN system. An ISBN-10 may also end with a capital X, which represents the value 10 when used as a check digit.

The entered ISBN is also compared with the ISBN-10 and ISBN-13 values already stored in the user’s Reading List. If a matching book is found, the program displays its index in the list and returns to the main menu. This helps prevent duplicate entries and unnecessary API requests.

2.2. Following Open Library Identifiers

The initial API response contains much of the book’s bibliographic information, but not necessarily everything needed for a complete record. It also provides Open Library Identifiers (OLIDs) that can be used for additional requests:

{
  "works": [
    {
      "key": "/works/OL26417584W"
    }
  ],
  "key": "/books/OL40329484M",
  "authors": [
    {
      "key": "/authors/OL9956826A"
    }
  ]
}

These identifiers point to three different levels of information:

  • - Work OLID: general information about the work
  • - Edition OLID: information about a specific edition
  • - Authors OLID: information associated with an author

The first search response does not include the author’s actual name. The program makes therefore an additional request using the Author OLID. In some cases, the author array may be empty. A query using the Work OLID is then required to retrieve the corresponding author identifiers.

3. API Usage and Error Handling

From the beginning of development, I took the Open Library API Usage Guidelines into account. In particular, the program identifies itself through a User-Agent header and tries to avoid duplicate or unnecessary requests.

The program also handles errors returned by Open Library’s servers and displays a specific message to help the user identify what went wrong.

4. Simplifying Date and Language Formats

As a general rule, the keys and values in the JSON object associated with a book are not modified during the first step of the program, the Reading List. Date and language values are exceptions because they are reformatted to facilitate later processing. Dates are converted into a consistent numeric YYYY-MM-DD format when possible. Open Library language identifiers may use three-letter codes such as "eng" and "fre"; for the Personal Library & Notebook, the program converts the languages it recognizes to two-letter codes such as "en" and "fr".

E. Applying Citation Style Language to Open Library Data

1. Why I Chose Citation Style Language

The Citation Style Language provides a common structure for bibliographic data across many fields of knowledge, including law, history, and the humanities. Its standardized fields help make bibliographic data easier to exchange between tools and provide a useful framework for long-term data preservation.

Although CSL was originally developed around XML, its JSON data model is also well documented: (a) CSL citation ; (b) CSL data. For this project, it provides a consistent structure for reorganizing the data retrieved from Open Library.

2. Adapting Open Library Data to CSL

Open Library’s data comes from a variety of sources, and some fields are not always represented in the same way. The contributions field is a good example.

2.1. Data About People

In the contributions field, names and roles may appear in several different forms.


{
  "contributions": [
    {
      "name": "Gildenhard, Ingo"
    }
  ]
}

{
  "contributions": [
    {
      "name": "Mark Kramer (Editor)"
    },
    {
      "name": "Wendy Call (Editor)"
    }
  ]
}

{
  "contributions": [
    {
      "name": "Bloch, Ernst, 1885-"
    }
  ]
}
  
{
  "contributions": [
    {
      "name": "John E. Woods (Translator)"
    }
  ]
}

Sometimes the information is better structured in the contributors field:


{
  "contributors": [
    {
      "role": "Translator",
      "name": "Paul Arhex"
    }
  ]
}

Information about people may appear under authors, contributions, or contributors. In addition, Open Library does not consistently separate given and family names, while CSL provides dedicated name fields and more specific roles such as editor and translator.

My approach is to automate this reformatting as much as possible while keeping in mind that human intervention is preferable to automatically introducing an error or inconsistency. Whenever the source data is clear enough, the program attempts to assign each person to a more appropriate CSL field.

For example, a contribution can be transformed into a structure such as:


{
  "translator": [
    {
      "family": "Arhex",
      "given": "Paule"
    }
  ]
}

Depending on the quality of the source data, the result may still need to be reviewed by the user.


{
  "contributor": [
    {
      "family": "Bloch",
      "given": "Ernst 1885-"
    }
  ]
}
2.2. Dates and the CSL Data Model

Dates turned out to be one of the less straightforward parts of the CSL data model. The CSL schema provides a structured date-parts representation as well as properties such as raw. Different tools may rely on different representations, so the JSON file used by the Personal Library & Notebook stores both:


{
  "issued": {
    "date-parts": [
      [
        1991,
        2,
        12
      ]
    ],
    "raw": "1991-02-12"
  }
}

This keeps a structured representation available while also retaining the simpler YYYY-MM-DD value already used elsewhere in the program.

F. Markdown & YAML Frontmatter Generation

The JSON file corresponding to the Personal Library & Notebook is designed to follow the CSL data model while retaining a small number of fields specific to this project, such as added and separate ISBN-10 and ISBN-13 values.

The program then generates YAML frontmatter for a Markdown file. At this stage, my main concern is no longer to reproduce every aspect of the JSON structure exactly, but to preserve the useful bibliographic information in a form that remains easy to read and edit.

A typical generated frontmatter block looks like this:

---
id: canetti1991-auto-da-fe
type: book
title: Auto-da-fe
author:
  - family: Canetti
    given: Elias
translator:
  - family: Arhex
    given: Paule
publisher: Gallimard
issued: '1991-02-12'
language: fr
ISBN-13: '9782070721825'
ISBN-10: '2070721825'
added: '2026-08-29'
---

For names, additional properties such as dropping-particle and non-dropping-particle can still be used when needed for more complex cases, such as “Jean de La Fontaine”, “Gabriele D’Annunzio”, or “Miguel de Cervantes”.

G. Project Context

This project was originally developed as my final project for Harvard Online’s CS50 Python course. Beyond the course itself, the project grew out of a practical question I had been considering for some time: how to combine a reading list, bibliographic information, and personal notes in a format that remains open, portable, and usable over the long term.

Working on it gave me an opportunity to bring together Python, API requests, JSON processing, bibliographic data, Markdown, YAML, and a problem I was already interested in: personal knowledge management.