Open-source developer infrastructure for internal tools (APIs, background jobs, workflows and UIs). Self-hostable alternative to Retool, Pipedream, Superblocks and a simplified Temporal with autogenerated UIs and custom UIs to trigger workflows and scripts as internal apps.
Scripts are turned into sharable UIs automatically, and can be composed together into flows or used into richer apps built with low-code. Supported script languages supported are: Python, TypeScript, Go, Bash, SQL, and GraphQL.
Try it - Docs - Discord - Hub - Contributor's guide
Windmill - Developer platform for APIs, background jobs, workflows and UIs
Windmill is fully open-sourced (AGPLv3) and Windmill Labs offers
dedicated instance and commercial support and licenses.
Main Concepts
-
Define a minimal and generic script in Python, TypeScript, Go or Bash that
solves a specific task. The code can be defined in the
provided Web IDE or
synchronized with your own GitHub repo
(e.g. through
VS Code
extension): -
Your scripts parameters are automatically parsed and
generate a frontend.
-
Make it flow! You can
chain your scripts or scripts made by the community shared on
WindmillHub. -
Build complex UIs on top of
your scripts and flows.
Scripts and flows can also be triggered by a
cron schedule (e.g.
'_/5 _ * * *') or through
webhooks.
You can build your entire infra on top of Windmill!
Show me some actual script code
CLI
We have a powerful CLI to interact with the windmill platform and sync your scripts from local files, GitHub repos and to run scripts and flows on the instance from local commands. See
more details.
Running scripts locally
You can run your script locally easily, you simply need to pass the right
environment variables for the wmill
client library to fetch resources and
variables from your instance if necessary. See more:
https://www.windmill.dev/docs/advanced/local_development.
To develop & test locally scripts & flows, we recommend using the Windmill VS
Code extension: https://www.windmill.dev/docs/cli_local_dev/vscode-extension.
Stack
- Postgres as the database.
- Backend in Rust with the following highly-available and horizontally scalable.
Architecture:- Stateless API backend.
- Workers that pull jobs from a queue in Postgres (and later, Kafka or Redis.
Upvote #173 if interested).
- Frontend in Svelte.
- Scripts executions are sandboxed using Google's
nsjail. - Javascript runtime is the
deno_core rust library (which itself uses
the rusty_v8 and hence V8 underneath). - TypeScript runtime is Bun and deno.
- Python runtime is python3.
- Golang runtime is 1.19.1.
Fastest Self-Hostable Workflow Engine
We have compared Windmill to other self-hostable workflow engines (Airflow,
Prefect & Temporal) and Windmill is the most performant solution for both
benchmarks: one flow composed of 40 lightweight tasks & one flow composed of 10
long-running tasks.
All methodology & results on our
Benchmarks
page.
Security
Sandboxing
Windmill can use nsjail. It is production
multi-tenant grade secure. Do not take our word for it, take
fly.io's one.
Secrets, credentials and sensitive values
There is one encryption key per workspace to encrypt the credentials and secrets
stored in Windmill's K/V store.
In addition, we strongly recommend that you encrypt the whole Postgres database.
That is what we do at https://app.windmill.dev.
Performance
Once a job started, there is no overhead compared to running the same script on
the node with its corresponding runner (Deno/Go/Python/Bash). The added latency
from a job being pulled from the queue, started, and then having its result sent
back to the database is ~50ms. A typical lightweight deno job will take around
100ms total.
Architecture
How to self-host
We only provide docker-compose setup here. For more advanced setups, like
compiling from source or using without a postgres super user, see
Self-Host documentation.
Docker compose
Windmill can be deployed using 3 files:
(docker-compose.yml, Caddyfile and a
.env) in a single command.
Make sure Docker is started, and run:
Go to http://localhost et voilà :)
The default super-admin user is: admin@windmill.dev / changeme.
From there, you can follow the setup app and create other users.
More details in
Self-Host Documention.
Kubernetes (k8s) and Helm charts
We publish helm charts at:
https://github.com/windmill-labs/windmill-helm-charts.
Run from binaries
Each release includes the corresponding binaries for x86_64. You can simply
download the latest windmill
binary using the following set of bash commands.
OAuth, SSO & SMTP
Windmill Community Edition allows to configure the OAuth, SSO (including Google
Workspace SSO, Microsoft/Azure and Okta) directly from the UI in the superadmin
settings. Do note that there is a limit of 10 SSO users on the community
edition.
Commercial license
To self-host Windmill, you must respect the terms of the
AGPLv3 license which you do not
need to worry about for personal uses. For business uses, you should be fine if
you do not re-expose Windmill in any way to your users and are comfortable with
AGPLv3.
To
re-expose any Windmill parts to your users
as a feature of your product, or to build a feature on top of Windmill, to
comply with AGPLv3 your product must be AGPLv3 or you must get a commercial
license. Contact us at ruben@windmill.dev if you have any doubts.
In addition, a commercial license grants you a dedicated engineer to transition
your current infrastructure to Windmill, support with tight SLA, and our global
cache sync for high-performance/no dependency cache miss of cluster from 10+
nodes to 200+ nodes.
Integrations
In Windmill, integrations are referred to as
resources and resource types.
Each Resource has a Resource Type that defines the schema that the resource
needs to implement.
On self-hosted instances, you might want to import all the approved resource
types from WindmillHub. A setup script will prompt
you to have it being synced automatically everyday.
Environment Variables
Run a local dev setup
See the ./frontend/README_DEV.md file for all
running options.
only Frontend
This will use the backend of https://app.windmill.dev but your own frontend
with hot-code reloading. Note that you will need to use a username / password login due to CSRF checks using a different auth provider.
In the frontend/
directory:
- install the dependencies with
npm install
(orpnpm install
oryarn
) - generate the windmill client:
- Run your dev server with
npm run dev
- Et voilà, windmill should be available at
http://localhost/
Backend + Frontend
See the ./frontend/README_DEV.md file for all
running options.
- Create a Postgres Database for Windmill and create an admin role inside your
Postgres setup.
The easiest way to get a working db is to run This will also avoid compile time issue with sqlx'squery!
macro - Install nsjail and have it accessible in
your PATH - Install deno and python3, have the bins at
/usr/bin/deno
and/usr/local/bin/python3
- Install caddy
- Install the lld linker
- Go to
frontend/
:npm install
,npm run generate-backend-client
thennpm run dev
- You might need to set some extra heap space for the node runtime
export NODE_OPTIONS="--max-old-space-size=4096"
- In another shell
npm run build
otherwise the backend will not find thefrontend/build
folder and will not compile. - In another shell
sudo caddy run --config Caddyfile
- Go to
backend/
:env DATABASE_URL=
RUST_LOG=info cargo run - Et voilà, windmill should be available at
http://localhost/
Contributors
Copyright
Windmill Labs, Inc 2023
No reviews found!
No comments found for this product. Be the first to comment!