FreshPrice — Development Story

CLIENT

About the client

AgriHarvest Co., Ltd., one of the country's leading food e-commerce operators.

AgriHarvest Co., Ltd. is a food e-commerce operator handling farm-direct vegetables, fruit, rice, seasonings and processed foods. To transform the grocery shopping experience, alongside its own sales channel it planned to launch a cross-marketplace comparison portal where "consumers can instantly compare prices against other EC sites." While the market had price comparison sites for books and electronics, a cross-marketplace portal specialized in fresh food barely existed.

Built in-house around the concept of "giving shoppers an experience where they can confidently judge the lowest price of the same food item on every purchase" — a consumer price comparison platform. A Python Scrapy + Scrapy-Redis distributed crawler infrastructure monitors 4 major marketplaces 24/7, a Node.js API delivers same-item matching, price history and alerts, and a Next.js web app implements the cross-marketplace comparison screen.

Monitoring scope

500K+ SKU

Target EC sites

4 Major EC (3 big + 1 grocery-specialist)

Crawl processing

1M+ requests/day

BUSINESS FLOW

STEP

Browse products

Search food by category or keyword

STEP

4-EC cross comparison

List price, shipping and stock for the same item

STEP

Decide using price history

Check the right time to buy with a 30-day trend chart

STEP

Buy or set an alert

Buy at the lowest price, or get notified at your target price

CHALLENGE

Design challenges to solve

Four structural requirements the platform had to meet.

None of these are technology-selection issues; they are business requirements tied directly to price freshness, data accuracy, operating cost and user retention.

① Without monitoring 4 EC sites 24/7, the lowest price can't be guaranteed

Each EC sitePrices and stock change by the day and by the hour. A once-a-day batch crawl shows stale prices and loses consumer trust.

② If same-item matching accuracy is low, you end up comparing the wrong prices

Product names were written inconsistently across EC sites (volume, origin, brand), andIf the "same item" judgment is wrong price comparison itself becomes meaningless.

③ Building the crawler infrastructure on fixed resources drives up operating cost

such as sale periods and adding new EC sites Crawl volume swings 10×+ depending on the season. Always sizing for peak volume worsens cost efficiency in normal times.

④ Slow price-drop alerts mean missing the right time to buy and churning

If time passes between detecting a price drop and notifying,the notification arrives after the sale has ended —and you end up losing consumer trust.

ARCHITECTURE

The architecture we built together

Turning the price comparison experience into an "unstoppable foundation" across four areas.

Built AgriHarvest's food price comparison platform in-house from scratch. A Python Scrapy + Scrapy-Redis distributed crawler infrastructure auto-scales on AWS EC2; a Node.js + Express API implements same-item matching, price history and alert delivery; and a Next.js web app provides the cross-marketplace comparison screen.

Area ①

Distributed crawler infrastructure

Challenge

Without monitoring 4 EC sites 24/7, the lowest price can't be guaranteed.

Solution

A distributed crawler swarm built with Scrapy + Scrapy-Redis

EC2 worker fleets pull tasks from a shared Redis queue, crawling in parallel 24/7 with 4 EC × 15 spiders.

Area ②

Same-item matching

Challenge

If same-item matching accuracy is low, you end up comparing the wrong prices.

Solution

Unify SKU variation with an ETL + normalization engine

Normalizes product name, volume, origin and brand via rules + Elasticsearch fuzzy matching, maintaining quality with a human-review UI.

Area ③

Auto-scaling

Challenge

Building the crawler infrastructure on fixed resources drives up operating cost.

Solution

Auto-scale crawler EC2 up and down by Redis queue length

A dedicated orchestrator watches the backlog and auto-starts/stops EC2, optimizing cost in both normal periods and during sales.

Area ④

Real-time notifications

Challenge

Slow price-drop alerts mean missing the right time to buy and churning.

Solution

Multi-channel instant delivery with Bull queue + Socket.io + SendGrid

On a price update, instantly extracts users who hit their target and notifies them within seconds via Push (Socket.io) + email (SendGrid).

TECH STACK

Frontend

Next.js

ReactM Material-UI / Ant Design📊 Chart.js

Backend

Node.js + Express

MongoDB

MySQL

Elasticsearch

Redis · Bull

Socket.io

Crawler / Infra

Python

Scrapy + Scrapy-Redis

AWS EC2 (auto-scale)

S3✉ SendGrid

IMPACT

The value the platform creates

AgriHarvest's "consumer shopping experience" accelerates.

This foundation continuously generates business outcomes — price freshness, purchase retention, operating cost and notification accuracy — not just technical metrics.

① A foundation that instantly judges "the lowest price for the same item" on a single screen

Reduces the hassle of jumping between 4 EC sites, giving consumers an experience of buying at the lowest price without hesitation every time.

② A design that ensures transparency of price judgment

A 30-day price-history chart lets consumers decide for themselves whether to "buy now or wait." Price transparency directly builds trust.

③ Auto-scaling that optimizes operating cost

Crawler EC2 auto-scales up and down with demand — staying stable during sales while keeping cost efficiency in normal times.

④ Bring users back to the app with price-drop alerts

Push + email alerts within seconds deliver a "reaches you before the sale ends" experience, contributing to ongoing gains in revisit rate and LTV.

Turning "I want to buy the same food at the lowest price"4-EC cross price comparison— solved.

AgriHarvest Co., Ltd., one of the country's leading food e-commerce operators.

Four structural requirements the platform had to meet.

Turning the price comparison experience into an "unstoppable foundation" across four areas.

AgriHarvest's "consumer shopping experience" accelerates.

Experience AgriHarvest's product in a working demo.