Jiayu's Blog

Analysing HDB Resale Flat Transactions With VisiData

VisiData is a quick way to run simple queries on CSV data without having to load the data into a database or use R, Python or Excel. We can find a resale flat prices dataset on Data.gov.sg. It contains transaction data over multiple time periods in CSV format:

Running Airflow Webserver on Cloud Run

Apache Airflow is, according to its website, a platform created by the community to programmatically author, schedule and monitor workflows. Airflow is deployed in several several components, two of which are the Scheduler and Webserver. DAG Serialization allows the Webserver to read DAGs entirely from the database instead of having to parse DAG files.

You might not need to scrape

When trying to extract data from websites, it's not always necessary to immediately turn to scraping techniques. Many contemporary websites are not fully server rendered and instead dynamically fetch data from backend services. By identifying these backend services and making requests to them directly, it's often possible to get more data than is actually displayed on the website, and in a more machine-readable form.

Jolang

Jo is a Lisp dialect that compiles directly to Go source code. yi-jiayu/jolang A Lisp dialect which compiles directly to Go source code https://github.com/yi-jiayu/jolang This is a fizzbuzz solution in Jo: (package main) (import "fmt") (func main () (for (define i 0) (< i 100) (inc i) (if (= 0 (% i 15)) (fmt.

Mahjong Party

I recently built Mahjong Party, an online multiplayer Singaporean mahjong game. Here's a screenshot of a game in progress: If you're interested, you can head over to https://mahjong.party/ and give the tutorial a try, or jump into a game immediately with some bots!

Singapore COVID-19 cases API

The MOH website is the authoritative source of information about the current COVID-19 situation in Singapore, but it's not exactly machine-readable. It doesn't seem like there's any official API either. For those trying to build their own dashboard or crunch some data, it might look like you have no choice but to do some data entry, scrape HTML or get data from a third-party source.

From Zero to Zebra in Prolog

Prolog is a logic programming language, which involves thinking about problems slightly differently from other programming languages. I recently picked it up to try solving a logic puzzle called the Zebra puzzle. First steps There are multiple implementations of Prolog. Two common implementations are GNU Prolog and SWI-Prolog.

Terraform Provider for Telegram

Over the weekend, I wrote a custom Terraform provider for Telegram: yi-jiayu/terraform-provider-telegram Terraform provider for Telegram https://github.com/yi-jiayu/terraform-provider-telegram Terraform is an infrastructure as code tool which allows you to declaratively specify your infrastructure and reproducibly set it up.

A Serverless Status Page

This week, I created a status page for my web properties: http://status.jiayu.io.s3-website.ap-southeast-1.amazonaws.com/ The page itself and the status history are served from S3. They were generated by Checkup, a tool for creating self-hosted status pages. A scheduled Lambda function runs Checkup every 5 minutes to perform health checks and stores the results in S3.

Handling "no process" errors in Elixir

Let's say we have a PID1 from somewhere: iex(1)> pid = :c.pid(0, 124, 0) #PID<0.124.0> And we want to use it as a GenServer reference: iex(2)> GenServer.call(pid, :do_something) ** (exit) exited in: GenServer.call(#PID<0.124.0>, :do_something, 5000) ** (EXIT) no process: the process is not alive or there's no process currently associated with the given name, possibly because its application isn't started (elixir) lib/gen_server.