Thursday, September 11, 2025

Hello, my web automation framework!

Recently I’ve been enjoying web automation. If you often find yourself repeating the same manual steps in the browser — logging in, clicking menus, filling forms, checking results — you know how tedious it can get. To save time, I built a lightweight web automation framework using Node.js and Puppeteer. The best part is that I can pause the automation at any step, do some manual work in the browser if needed, and then continue with the following steps seamlessly.

Here are key design ideas:

1. Store inputs in .env
Sensitive values like username, password, and target URLs don’t belong in code. I load them at runtime with dotenv:
/* .env
USER=myuser
PASS=secretpassword
URL=https://example.com
*/

import dotenv from "dotenv";
dotenv.config();

console.log(process.env.USER, process.env.URL);
2. Share state with a global context object
Instead of passing around dozens of variables, I keep everything in one ctx object. This context is passed into all modules and steps, so they can use common objects like browser, page, or config flags.

3. Reuse or create a browser page
The framework can attach to an existing page of the Chrome session if it’s already there, or launch a new one if not. This way I can debug in a real browser window or let the script create a fresh page for me.

4. Define steps as reusable objects
All actions are defined in ctx.steps, where each step has a name and a run(ctx) function. For example:
// app.mjs
import { runner } from "../runner.mjs";

ctx.steps = [
  {
    name: "Page step 1",
    run: async (ctx) => {
      const { page } = ctx;
      // do something on page
    },
  },
  {
    name: "Page step 2",
    run: async (ctx) => {
      const { page } = ctx;
      // do something on page
    },
  },
];

// kick off the web automation
await runner(ctx);

5. Focus only on the steps.
With this framework, I just focus on writing the steps. And since they’re modular, I can run them sequentially, skip some, or even execute them in random order — all with the same framework.
Usage:
    node app.mjs                     # run all steps
    node app.mjs 3                   # run from step 3 to end
    node app.mjs 2 5                 # run steps 2..5
    node app.mjs --list              # list steps


In the future, I plan to extend this framework with screenshots on errors, step runtime tracking, and detailed reporting. That way, not only will the automation save me from repetitive work, but it will also give me clear insights into how each step performs and where failures happen.

No comments: