Hacker News
App overview
Hacker news app allows reading the news in a custom interface. The interface is pretty simple and consists of the main and story pages.
On the main page, you can choose what stories to load. Newest (default) or Hottest. You can also pick the number of stories to display, and refresh the page with the corresponding button.
The story page contains the story itself with credentials (author, published time), comments tree, and comments counter.
As simple as it looks, the way the app works under the hood is a little bit tricky. And the reason for this is the API.
API
Hacker news has a very special API. One would expect to make one request for a blog post and receive a friendly response with all the post content, detail, comments count, and the tree of comments.
But that would have been too easy 🙂
As stated on the API readme:
The v0 API is essentially a dump of our in-memory data structures. We know, what works great locally in memory isn’t so hot over the network. Many of the awkward things are just the way HN works internally. Want to know the total number of comments on an article? Traverse the tree and count. Want to know the children of an item? Load the item and get their IDs, then load them. The newest page? Starts at item maxid and walks backward, keeping only the top level stories. Same for Ask, Show, etc.
Stories, comments, jobs, Ask HNs and even polls are just items. They’re identified by their ids, which are unique integers, and live under
/v0/item/<id>.
So, every item is an entity (job, story, comment, poll, or pollopt) with id as required filed, and a bunch of optional properties.
Taking this into consideration a simple task of displaying comments, turns into traversing the whole tree of comments and making lots of requests to the server. Thankfully, there is no rate limit on how many requests we can make 😮💨
Dive into arhitecture
I’m using React for views, Redux for storing data, Sass for styling, and Axios for API requests. Below I will go through detailed implementations of main features.
Main page: Newest and Hottest
Depending on the value chosen in the stories picker we make a request to /v0/newstories or /v0/topstories.
Response is the array of item id’s, which we slice according to the value chosen by the user in the picker.
[35280583,35280571,35280569,35280540,35280475,...]All picker values are saved in the global store.
Then for all the id’s we fetch the stories using Promise.all.
function getNewsItem(itemId) {
return axios.get(`${baseURL}/item/${itemId}.json`)
.then((res) => res.data);
}
async function getItemsByIds(itemIds) {
return await Promise.all(itemIds.map(getNewsItem));
}Before storing the stories in the store we populate the story object with a formatted publish date because the original story has time defined in Unix time: "time": 1175714200.
const getDateTime = (timestamp) => {
const date = new Date(timestamp * 1000);
return date.toISOString();
}
const getHumanReadableTime = (timestamp) => {
const date = new Date(timestamp * 1000);
return `${date.toLocaleDateString()} ${date.toLocaleTimeString()}`;
}
const populateItemsWithTime = (items) => {
items.forEach(item => {
item.dateTime = getDateTime(item.time);
item.humanReadableTime = getHumanReadableTime(item.time);
});
return items;
}
export const fetchNews = createAsyncThunk('news/fetchNews', async (arg, { getState }) => {
...
return populateItemsWithTime(newsItems);
});dateTime is the standard Date format that we are going to use for better semantics inside the <time> tag. humanReadableTime is what we are going to use to display the published date.
Refresh button will just repeat the request with the current picker’s values.
Story page: comments tree traversal
When the user navigates to the story page https://arbuznik-hacker-news.netlify.app/35281333 we render the body of the story from the store (if navigation happened from the main page) or from API (if user typed the direct URL in the browser).
The story body looks like this:
{
"by": "bryanh",
"descendants": 413,
"id": 35277677,
"kids": [
35280040, 35278215, 35278369, ...,
],
"score": 937,
"time": 1679590648,
"title": "ChatGPT Plugins",
"type": "story",
"url": "https://openai.com/blog/chatgpt-plugins"
}In the kids property, we get all the top-level comments. If we fetch the first comment, we get its credentials and kids.
{
"by": "mk_stjames",
"id": 35280040,
"kids": [
35280605, 35280374, ...,
],
"parent": 35277677,
"text": "I have some odd feelings about this. ...",
"time": 1679599613,
"type": "comment"
}Until there are no more kids, we keep fetching, using the BFS traversal algorithm in our Redux Async Thunk.
export const fetchComments = createAsyncThunk('news/fetchComments', async (arg, { getState }) => {
const state = getState();
// Get all the kids of our post
const firstLevelCommentsIds = state.news.item.kids;
const itemId = state.news.item.id
// Starting queue of top-level comments that we have from initial post
let queue = [...firstLevelCommentsIds];
// Keep track of all the comments we've visited
let visited = new Set(queue);
let allComments = [];
// We keep traversing the tree of comments until the queue is not empty
while (queue.length > 0) {
// Get all the comments in queue
const currentCommentsStack = await getItemsByIds(queue);
// Reset the queue
queue = [];
// Add newly fetched comments to the main array
allComments = [...allComments, ...currentCommentsStack];
// Get all the kids comments id's in newly fetched comments and transform them into array of string ids
let childComments = currentCommentsStack
.filter(currentComment => currentComment.kids)
.map(currentComment => currentComment.kids)
.flat();
// If there are any kids, and we haven't visited them yet, mark them as visited and add them to the queue. Do one more iteration of traversing.
if (childComments) {
childComments.forEach(childComment => {
if (!visited.has(childComment)) {
visited.add(childComment);
queue.push(childComment);
}
})
}
}
// At the end: filter out all dead and deleted comments
const aliveComments = allComments.filter(comment => !comment.dead && !comment.deleted);
// Add human readable time
const aliveCommentsWithTime = populateItemsWithTime(aliveComments)
// Return the result
return { itemId, comments: aliveCommentsWithTime };
})Story page: recursive React components
Rendering happens inside two components, <CommentsTree> and <Comment>. <CommentsTree> is our main render component. This is how it looks.
// Pass parentId as props
const CommentsTree = ({ parentId }) => {
// Get all the comments from store
const { comments, itemId } = useSelector(selectNewsItemComments);
// If there's no parentId, then the comments tree mounts to the top level. Parent is story itself.
if (!parentId) {
parentId = itemId;
}
// Don't render if there are no comments at all
if (!comments) {
return null;
}
// Get all comments for given level
const commentsToRender = comments.filter(comment => {
return comment.parent === parentId;
})
// Recursion breaker if there are no more child comments
if (!commentsToRender.length) {
return null;
}
return (
<ul className={styles.commentsTree}>
// Render all the comments for given level and inside it recursively render a new comments tree with current comment as a parent
{commentsToRender.map(item => (
<Comment key={item.id} item={item}>
<CommentsTree parentId={item.id} />
</Comment>
))}
</ul>
)
}<Comment>
const Comment = ({ item, children }) => {
return (
<li className={styles.comment}>
{item.text && <p className={styles.text} dangerouslySetInnerHTML={{ __html: item.text }}/>}
{item.by && <p className={styles.author}>By {item.by}</p>}
{item.time && <p className={styles.date}><time dateTime={item.dateTime}>{item.humanReadableTime}</time></p>}
// Children is our nested <CommentsTree>
{children}
</li>
)
}As a result, our comments are rendered in beautifully organized threads.
Summary
This looked like a straightforward project at first, but given the peculiarities of the Hacker News API, I had the chance to practice my tree traversal knowledge. Recursive components were a fun part, I really enjoyed coding them.
Feel free to check the app and source code.
Comments