Endless data possibilities: Utilizing APIs for import into Kumu
How we used the Spotify API to visualize data on music playlists for this year’s Black History Month.
Recently, I was asked by a Kumu user to create a map visualization based on data from a third-party application. Too bad that this external tool didn’t have a way to export data into a Google Sheet, so I could easily import it into Kumu. As I was not looking forward to the tedious work of having to manually bring in all that data, I looked into using an API.
The acronym “API” was a term I was familiar with, but not a concept I had ever explored for my own Kumu maps. So when I finally dove into using an API to gather data, I was delighted to find out that they: 1) aren’t as scary as I thought—especially because a lot of them have great documentation, and 2) I wouldn’t need any sort of programming interface to access the data, as I simply used a Google Sheets plugin that takes your request and outputs the data neatly into a Google Sheet.
Let’s start with a quick intro: an API, which stands for Application Programming Interface, is like a bridge between you and the information you’d like to obtain from a particular external application. It’s kind of how someone taking your order at a restaurant might convey what you want to the kitchen: you tell them what you want to eat, and they’ll have a specific button on the register to send the information of your order to the kitchen, in a way the kitchen can quickly process all incoming orders. Fast forward to 10 minutes later, and you’ll have a steaming plate of yummy food in front of you.
Since my first encounter with an API was so successful, I wanted to create more visualizations based on data from other platforms. The possibilities suddenly seemed endless!
I was reminded that this year’s Black History Month was about celebrating African Americans and the Arts, so I wanted to explore the artistic pursuits of African Americans in my favorite art genre: music. It was a no-brainer, time to explore data visualizations using Spotify’s API!
Making the Map
For this Kumu project, I was interested in visualizing the top 40 playlists for Black History Month¹, specifically in the US market, and what songs they contained. Find the steps I took to create this map below!
Step 0: Find a tool to connect your API to a Google Sheet
I used API Connector, a Google Sheets plugin, but I’m sure there’s more out there.
Step 1: Get a list of the top 40 playlists in Black History Month
The way Spotify’s API works is that it’s an actual URL where you can add parameters to search for certain data. Now, I need to take a moment here and convey an important message I received from one of our Kumu developers as I was writing this blog:
“Not all APIs are HTTP/web based. Really, an API is what it gets called when there’s any kind of programmable interface to access an application’s data. Therefore, the plugin you’re using might not be able to work / retrieve data from all possible platforms you have in mind”
Consider yourself warned! Either way, in this specific case (and many others) Spotify’s documentation actually helps you build the “request”, as they call it. You choose from different “endpoints” based on what data you’d like, and then choose among the parameters. The request I used to grab the list of playlists is:
https://api.spotify.com/v1/search?query=black+history+month&type=playlist&market=US&locale=en-US%2Cen%3Bq%3D0.9&offset=0&limit=40First, I created a new spreadsheet in Google Sheets, and started up the API Connector extension. Then, I created a new custom API request in their interface, and copied in my request where it says Request URL.
They have a spot where you authenticate yourself (you register yourself with your Spotify login to use the API and then get the authentication through them).
Lastly, I selected the cell I’d like the data to output to under Output Settings , and then I was able to run the request! The results look something like this:
Step 2: Get the list of tracks in the top playlists
Now that I had a list of top playlists, these came with an ID. And the Spotify API has a way of using that playlist ID, to then get a list of tracks contained in that playlist!
The request looks like this:
https://api.spotify.com/v1/playlists/PLAYLIST_ID_HERE/tracksSince I needed to run this request for 40 playlists to get all the tracks on each playlist, what I did was grab the playlist IDs and put them into a new tab in my spreadsheet, and then used the =TEXTJOIN function to create a column for a request URL for each playlist!
The cool thing about the API Connector is that you can paste more than one request to run at a time, so I simply pasted in the list of 40 requests with each playlist ID into the Request URL section.
In Output Options , I made sure to check these, so that I could see where in the outputted data a new playlist’s list of tracks was starting:
After a lot of minutes of datacrunching, I had a list of 2,021 tracks with their metadata: the artist, album images, popularity², and even a track preview link. I was very satisfied with the results so far!
Step 3: Format the data for Kumu
Now, it was time to structure the data for import into Kumu. I rearranged all the data into the standard Kumu format, with columns for the Labels and Type, and then connected the Google Sheet to my Kumu project.
Step 4: Refine the Kumu visualization of the playlists and tracks
At this point, there weren’t any connections drawn between playlists and the tracks that are on them, because we only imported a list of tracks and playlists, not what’s connected to what.
To work around this, I used Kumu’s “Connect by” feature to select the Playlist field, because that is data I had listed for each track. As such, Kumu finds the elements that are playlists and automatically connects them to the elements that are tracks.
Then, for decoration purposes, I chose the shape of the playlists to be circles, and the tracks to be triangles. Furthermore, tracks are sized by degree, for which I used Kumu’s Metrics functionality to calculate that number. The higher the degree, the more playlists a track appears on!
Lastly, the element profile on the left allows you to play a track preview, and the tracks inherit the album image url that the Spotify API gave us!
Some other features I used for this map:
- I added popovers, so that you can hover over a track or playlist and see its name and basic info,
- I added controls at the top, if you want to filter the data to see only the songs that appear across many playlists.
- I used the selector builder in the search bar (rocket icon off to the right after clicking on the Search bar) to select only playlists, and then used the snap-to feature to put those playlists in a circle.
Besides the documentation I’ve linked to, you can also visit the map here, and look at the settings to see these features in action.
That’s all folks! Let me know if you have any questions, and reach out to support@kumu.io if you need any help making your own visualizations!
Footnotes
1. Information I could gather about Playlist search results:
- “I believe the results are sorted the same way as they are in the spotify app itself, which I believe is by relevance, so by name and popularity.” — Spotify Community Member
- “It’s usually related to songs in a playlist, or the name of the playlist.”
- “I wonder about this too, since playlists with tons of followers usually come up first. That makes me think it’s based on relevance and popularity.” — Spotify Community Members
- “I think it only searches the title and maybe the description? Could also be some sort of metadata (followers, tags, etc) for specific playlists (like Spotify’s playlists). They could also be personalizing the search. But I’m just guessing. I did notice that searching in the Spotify app sometimes yields different results than with their search API, even when using the same query.” — Spotify Redditor
2. Information on track popularity:
- “The popularity is calculated by algorithm and is based, in the most part, on the total number of plays the track has had and how recent those plays are.
Generally speaking, songs that are being played a lot now will have a higher popularity than songs that were played a lot in the past.” — API Documentation - “Spotify’s popularity is indeed based on the number of streams, but instead of total plays, it’s based on a short timeframe. This is why you’ll often see a more popular number on #1 on an artist profile that has less plays than the #2.” — Stack Overflow Contributor
