Transform RSS Feed into API using Express JS

Home Author Contact

NodeJS

Laurent Egbakou

February 23, 2022

2 min

Our work may have helped you ?

Support Us

Required tools

Start a new NodeJS project

Install Express.js and rss-parser

This blog post will cover how to build crypto news API based on RSS Feed using Express JS. The article will include how to fetch RSS Feed and create a route that returns news.

RSS is simply an XML text file created or automatically generated by a website publisher. It contains a running list of articles or other content published by the site, with the newest entry always at the top of the list. Its name is an acronym for Really Simple Syndication.

Required tools

To follow the steps in this article, you'll need the following tools:

Node.js & npm - includes the Node.js runtime, command-line tools, and package manager, install it from https://nodejs.org/.
Visual Studio Code or WebStorm. It doesn't matter which one but I use WebStorm.
Cryptocurrency RSS Feeds (only one, choose the feed you want. I'll use the Bitcoin news RSS feed):
- Cointelegraph RSS Feed: https://cointelegraph.com/rss
- NewsBTC RSS Feed: https://www.newsbtc.com/feed
- Bitcoin.com News RSS Feed: https://news.bitcoin.com/feed/
- Coinbase Blog RSS Feed: https://blog.coinbase.com/feed

Start a new NodeJS project

Before starting any new Node.js project we should run npm init to create a new package.json file for our project.

Create a new empty directory and run npm init. You'll then answer a few basic questions about your project, and npm will create a new package.json file for you when you're done.

mkdir rss-to-express-api
cd crypto-news-api
npm init
This utility will walk you through creating a package.json file.
It only covers the most common items, and tries to guess sensible defaults.
See `npm help init` for definitive documentation on these fields
and exactly what they do.
Use `npm install <pkg>` afterwards to install a package and
save it as a dependency in the package.json file.
Press ^C at any time to quit.
package name: (rss-to-express-api)
version: (1.0.0)
description: Transform RSS Feeds into API
entry point: (index.js) app.js
test command:
git repository: https://github.com/lioncoding-oss/rss-to-express-api.git
keywords:
author: Laurent Egbakou
license: (ISC) MIT
About to write to /home/lioncoding-oss/rss-to-express-api/package.json:
{
"name": "crypto-news-api",
"version": "1.0.0",
"description": "Transform RSS Feeds into API",
"main": "app.js",
"scripts": {
"test": "echo \"Error: no test specified\" && exit 1"
},
"author": "Laurent Egbakou",
"license": "MIT"
}
Is this OK? (yes) yes

Install Express.js and rss-parser

We’re almost ready to write our Express JS API, but first, we need to install the express and rss-parser packages using npm. RSS-parser is an npm package to fetch RSS Feeds.

# Install Express JS and Rss-parser
npm install express rss-parser
# List the installed dependencies and their versions
npm list --depth=0
[email protected] /home/lioncoding-oss/crypto-news-api
+-- [email protected]
`-- [email protected]

Let's code

Before starting to code, we need to identify the information we want to get from the Bitcoin RSS feed. For every new in the feed, we would like to retrieve the title, the link, and the publication date.

Now let's create the entry point we specified when using the npm init command. In my case, the entry point is app.js.

const express = require('express')
const app = express()
app.get('/api/ping', (req,res) => {
var response = { answer: "pong" }
res.status(200).json(response)
})
app.listen(3000, () => {
console.log("Server is running on port 3000")
})

Run the Express JS API and test the ping endpoint by using the following commands:

node app.js: start the express js app
curl http://localhost:3000/api/ping : must return {"answer":"pong"}

Configure the rss-parser package

let Parser = require('rss-parser')
let parser = new Parser({
headers: { 'User-Agent': 'Chrome' }
});

Create a function to read any RSS Feed

async function fetchRssFeed(feedUrl) {
let feed = await parser.parseURL(feedUrl);
return feed.items.map(item => {
return {
title: item.title,
link: item.link,
date: item.pubDate
}
});
}

Create a route that returns news

const bitcoinFeedUrl = 'https://news.bitcoin.com/feed/'
app.get('/api/news', async (req, res) => {
await fetchRssFeed(bitcoinFeedUrl)
.then(data => {
res.status(200).json(data)
})
.catch(err => {
res.status(404).json({
status: 'error',
message: 'An error occurred when fetching news'
})
})
})

Here is the complete code of the app.js file

const express = require('express')
let Parser = require('rss-parser')
const app = express()
let parser = new Parser({
headers: { 'User-Agent': 'Chrome' }
});
const bitcoinFeedUrl = 'https://news.bitcoin.com/feed/'
async function fetchRssFeed(feedUrl) {
let feed = await parser.parseURL(feedUrl);
return feed.items.map(item => {
return {
title: item.title,
link: item.link,
date: item.pubDate
}
});
}
app.get('/api/news', async (req, res) => {
await fetchRssFeed(bitcoinFeedUrl)
.then(data => {
res.status(200).json(data);
})
.catch(err => {
res.status(500).json({
status: 'error',
message: 'No news found'
})
})
})
app.get('/api/ping', (req, res) => {
var response = { answer: "pong" }
res.status(200).json(response)
})
app.listen(3000, () => {
console.log("Server is running on port 3000")
})

Restart the app and test the endpoint /api/news

Make the API request using POSTMAN or cUrl. Bellow is the response from our API:

[
{
"title": "FTX Launches Gaming Unit to Offer Crypto Services to Other Companies",
"link": "https://news.bitcoin.com/ftx-launches-gaming-unit-to-offer-crypto-services-to-other-companies/",
"date": "Wed, 23 Feb 2022 08:30:11 +0000"
},
{
"title": "Kazakhstan Cracks Down on Illegal Mining, Busts 13 Crypto Farms",
"link": "https://news.bitcoin.com/kazakhstan-cracks-down-on-illegal-mining-busts-13-crypto-farms/",
"date": "Wed, 23 Feb 2022 06:30:03 +0000"
},
{
"title": "Novatar: The Ultimate NFT Project That Is Dominating the NFT Market Now",
"link": "https://news.bitcoin.com/novatar-the-ultimate-nft-project-that-is-dominating-the-nft-market-now/",
"date": "Wed, 23 Feb 2022 06:00:34 +0000"
},
{
"title": "El Salvador’s Tourism Rises 30% After Bitcoin Became Legal Tender",
"link": "https://news.bitcoin.com/el-salvadors-tourism-rises-30-after-bitcoin-became-legal-tender/",
"date": "Wed, 23 Feb 2022 04:00:22 +0000"
}
]