Documentation
Jaypore CI: Minimal, Offline, Local CI system.
Install
Download the binary for your platform from www.jayporeci.in. Available builds:
| Platform | Binary |
|---|---|
| Linux x86-64 | git-jci-linux-amd64 |
| Linux ARM64 | git-jci-linux-arm64 |
| Linux ARMv7 | git-jci-linux-armv7 |
| Linux 32-bit | git-jci-linux-386 |
| Windows x86-64 | git-jci-windows-amd64.exe |
| Windows ARM64 | git-jci-windows-arm64.exe |
macOS is not currently offered as a pre-built binary. You can build from source with
go build ./cmd/git-jci.
After downloading, move the binary to somewhere on your PATH:
sudo mv git-jci-linux-amd64 /usr/local/bin/git-jci
The binary is fully static (no dependencies) and works on most systems. Once installed, git will automatically find it as a subcommand and you can use it via git jci.
Config
Create a .jci folder in your repository and add a run.sh file. Optionally add a crontab file for scheduled runs.
.jci/
├── run.sh # required: defines your CI pipeline
└── crontab # optional: schedule recurring runs
Make run.sh executable:
chmod +x .jci/run.sh
run.sh can be anything ; a shell script, a wrapper that calls a Python program, a binary, a docker-compose invocation ; whatever you like.
Scheduling with crontab
.jci/crontab uses standard 5-field cron syntax with two optional keyword arguments:
# SCHEDULE [branch:BRANCH] [name:NAME]
0 0 * * * # midnight, current branch
0 0 * * * branch:main # midnight, main branch
*/15 * * * * name:quick-check # every 15 min, named job
0 0 * * * branch:main name:nightly # named midnight build on main
After editing .jci/crontab, run git jci cron sync to install the entries into your system’s crontab. Each installed entry runs git-jci run from the repository root. For example, the branch:main name:nightly entry above becomes:
0 0 * * * cd '/path/to/repo' && git fetch --quiet 2>/dev/null; git checkout --quiet main 2>/dev/null && git pull --quiet 2>/dev/null; git-jci run # JCI:<repoID> [nightly]
Cron jobs run with the repository root as the working directory and have access to all three JCI_* environment variables.
Environment Vars
Your run.sh script receives these environment variables:
| Variable | Description |
|---|---|
JCI_COMMIT |
Full commit hash |
JCI_REPO_ROOT |
Absolute path to the repository root |
JCI_OUTPUT_DIR |
Absolute path to the output directory for this run |
All three are always set. The script’s initial working directory is JCI_REPO_ROOT. Any files written to JCI_OUTPUT_DIR become CI artifacts , browsable and downloadable via git jci web. The output directory is a temporary directory that is cleaned up after its contents are stored in git.
Non-zero exit = failure. If
run.shexits with a non-zero code, the run is marked as failed (✗ err). The artifacts are still saved and viewable.
Example workflow
# 1. Enter the repository
cd my-project
# 2. Create the CI script (only needed once)
mkdir -p .jci
cat > .jci/run.sh << 'EOF'
#!/usr/bin/env bash
set -euo pipefail
cd "$JCI_REPO_ROOT"
echo "Running tests..."
go test ./... | tee "$JCI_OUTPUT_DIR/test-results.txt"
EOF
chmod +x .jci/run.sh
# 3. Commit and run CI
git add -A
git commit -m "add CI"
git jci run # execute .jci/run.sh and store results for this commit
# 4. View results locally (opens at http://localhost:8000)
git jci web
# 5. Share results with teammates
git jci push # push CI refs to origin
git jci pull # fetch CI refs pushed by teammates
# 6. Maintenance
git jci prune # dry-run: show what would be removed
git jci prune --commit # actually delete CI refs for gone commits
git jci prune --commit --older-than=14d # also delete refs older than 14 days
# 7. Cron
git jci cron ls # list scheduled jobs in .jci/crontab
git jci cron sync # install/update entries in the system crontab
You can also trigger git jci run automatically via git hooks so CI runs on every commit without any manual step.
How it works
CI results are stored as git tree objects under the refs/jci/ namespace (or refs/jci-runs/ when a commit has multiple runs). This keeps them completely separate from your regular branches and tags while still being part of the repository.
- Results are never checked out to the working directory
- They are pushed and pulled exactly like any other git refs but only when you explicitly run
git jci push/git jci pull. A plaingit pushdoes not send them. - They are garbage collected when the original commit is gone (via
prune) - Each run produces a
run.output.txtartifact plus whatever else yourrun.shwrites toJCI_OUTPUT_DIR
Viewing results
git jci web starts a local web server at http://localhost:8000. It shows all commits on your branches, their CI status, and lets you browse and download artifacts. Clicking a commit shows its output files in a three-panel layout:

You can pass a custom port: git jci web 9000.
Sharing results with teammates
# You: after running CI
git jci push # pushes refs/jci/* to origin
# Teammate: to see your results
git jci pull # fetches refs/jci/* from origin
git jci web # now shows your CI results too
CI refs are pushed to and pulled from origin by default. You can specify a different remote: git jci push my-remote.
FAQ / Needs / Wants / Todos
- Complex pipeline definitions
run.shcan be any executable. Do whatever you like!
- Artifacts
- Anything written to
JCI_OUTPUT_DIRis an artifact, browsable viagit jci web.
- Anything written to
- Debug CI locally
- Just run
.jci/run.shdirectly. Set the three env vars yourself to simulate a CI environment.
- Just run
- Automate unit, integration, and end-to-end test suites on every commit
- Run linting and static analysis to enforce coding standards
- Link git hooks and run CI whenever you want.
- Produce code coverage reports and surface regressions
- Write HTML coverage reports to
JCI_OUTPUT_DIRand view them viagit jci web. - For regressions,
run.shcan commit example files created by tools like hypothesis back to the repo. This ensures that next runs will use those examples and test for regressions.
- Write HTML coverage reports to
- Build, package, and archive release artifacts across target platforms
- I like building a docker image, building stuff inside that, then publishing.
- See
.jci/run.shin this repository for a real-world cross-compile + publish example.
- Perform dependency and source code security scans (SCA/SAST)
- I like to run Truffle Hog to prevent accidental leaks.
- Generate documentation sites and preview environments for review
- This Jaypore CI site itself is generated and published via CI.
- Schedule recurring workflows (cron-style) for maintenance tasks
- I run a nightly build via cron to ensure I catch dependency failures / security breaks. See .jci/crontab for an example.
- Notify developers and stakeholders when CI statuses change or regress
- As part of our scripts, we can call telegram / slack / email APIs and inform devs of changes.
- Built-in secrets management with masking, rotation, and per-environment scoping
- I currently use Mozilla SOPS for secrets but this might change in the future.
- Build farms / remote runners on cloud
- Community / marketplace runners contributed by external teams
- Shared runner pools across repositories and organizations
- Deploy keys / scoped access tokens so runners can securely pull & push repos
- Merge request / PR status reporting, required-check gating, and review UIs
- It would be great to have some integration into PRs so that we can know if our colleagues have run CI jobs or not.
- Line-by-line coverage overlays and annotations directly on PR/MR diffs
- This might be hard since it will depend a LOT on which remote is being used. Gitlab uses a cobertura file but others might not.
- Deployment environments with history, approvals, and promotion policies
- First-class integration with observability / error tracking tools (e.g., Sentry)
- Ecosystem of reusable actions/tasks with versioned catalogs and templates
- Validate infrastructure-as-code changes and deployment pipelines via dry runs
Examples
Golang Lint Build Test
$ tree .
.
├── .jci
│ └── run.sh
├── README.md
├── main.go
└── main_test.go
$ cat .jci/run.sh
#!/usr/bin/env bash
set -euo pipefail
# ── Navigate to the project directory ────────────────────────────────────────
cd "$JCI_REPO_ROOT/00-golang-lint-build-test"
echo "==> Working directory: $(pwd)"
echo "==> Commit: ${JCI_COMMIT:-unknown}"
echo
# ── Initialise Go module if needed ───────────────────────────────────────────
if [ ! -f go.mod ]; then
echo "==> No go.mod found – initialising module"
go mod init example.com/server
fi
# ── Lint (go vet) ────────────────────────────────────────────────────────────
echo "==> Running go vet ./..."
go vet ./... 2>&1 | tee "$JCI_OUTPUT_DIR/vet-report.txt"
echo " vet report saved to \$JCI_OUTPUT_DIR/vet-report.txt"
echo
# ── Format check ─────────────────────────────────────────────────────────────
echo "==> Checking formatting with gofmt"
unformatted=$(gofmt -l .)
if [ -n "$unformatted" ]; then
echo " ERROR: the following files need gofmt:"
echo "$unformatted"
exit 1
fi
echo " all files formatted correctly"
echo
# ── Build ────────────────────────────────────────────────────────────────────
echo "==> Building binary"
go build -o "$JCI_OUTPUT_DIR/server" .
echo " binary saved to \$JCI_OUTPUT_DIR/server"
echo
# ── Test ─────────────────────────────────────────────────────────────────────
echo "==> Running tests"
go test -v -cover ./... 2>&1 | tee "$JCI_OUTPUT_DIR/test-results.txt"
echo " test results saved to \$JCI_OUTPUT_DIR/test-results.txt"
echo
# ── Publish (optional) ───────────────────────────────────────────────────────
if [ -n "${PUBLISH_DIR:-}" ]; then
echo "==> Publishing binary to $PUBLISH_DIR"
mkdir -p "$PUBLISH_DIR"
cp "$JCI_OUTPUT_DIR/server" "$PUBLISH_DIR/server"
echo " published successfully"
else
echo "==> PUBLISH_DIR not set – skipping publish step"
fi
echo
# ── Summary ──────────────────────────────────────────────────────────────────
echo "========================================"
echo " Pipeline complete"
echo " Commit : ${JCI_COMMIT:-unknown}"
echo " Binary : $JCI_OUTPUT_DIR/server"
echo " Vet : $JCI_OUTPUT_DIR/vet-report.txt"
echo " Tests : $JCI_OUTPUT_DIR/test-results.txt"
echo "========================================"
Pylint Pytest Coverage
$ tree .
.
├── .jci
│ └── run.sh
└── README.md
$ cat .jci/run.sh
#!/bin/bash
set -euo pipefail
# Jaypore CI run script
# ---------------------
# This script is executed by Jaypore CI.
#
# Available environment variables:
# JCI_COMMIT - The git commit being tested
# JCI_REPO_ROOT - Absolute path to the repository root
# JCI_OUTPUT_DIR - Directory for CI artifacts (cwd at start)
#
# Any files written to JCI_OUTPUT_DIR become CI artifacts.
echo "=== Jaypore CI: Pylint + Pytest + Coverage ==="
echo "Commit : $JCI_COMMIT"
echo "Repo : $JCI_REPO_ROOT"
echo "Output : $JCI_OUTPUT_DIR"
echo
cd "$JCI_REPO_ROOT"
# ---- 1. Pylint ----
echo "--- Running Pylint on core/ ---"
DJANGO_SETTINGS_MODULE=mysite.settings \
pylint core/ \
--output-format=text \
--disable=C0114,C0115,C0116 \
| tee "$JCI_OUTPUT_DIR/pylint-report.txt" \
|| true # don't fail the build on lint warnings
echo
# ---- 2. Pytest + Coverage ----
echo "--- Running Pytest with Coverage ---"
pytest \
--cov=core \
--cov-report=html:"$JCI_OUTPUT_DIR/htmlcov" \
--cov-report=term \
| tee "$JCI_OUTPUT_DIR/pytest-results.txt"
echo
# ---- 3. Summary ----
echo "=== Summary ==="
echo "Pylint report : pylint-report.txt"
echo "Pytest output : pytest-results.txt"
echo "Coverage HTML : htmlcov/index.html"
echo "All artifacts are in $JCI_OUTPUT_DIR"
Docker Compose Api Tests
$ tree .
.
├── .jci
│ └── run.sh
├── Dockerfile
├── README.md
├── docker-compose.yml
└── test_api.sh
$ cat .jci/run.sh
#!/bin/bash
set -e
echo "=== Jaypore CI: Docker Compose API Tests ==="
echo "Commit: $JCI_COMMIT"
echo "Repo root: $JCI_REPO_ROOT"
echo "Output dir: $JCI_OUTPUT_DIR"
PROJECT_DIR="$JCI_REPO_ROOT/02-docker-compose-api-tests"
cd "$PROJECT_DIR"
COMPOSE_PROJECT="jci-api-tests-$$"
cleanup() {
echo "=== Cleaning up ==="
docker compose -p "$COMPOSE_PROJECT" down -v --remove-orphans 2>/dev/null || true
}
trap cleanup EXIT
# ---- Start services ----
echo "=== Starting services ==="
docker compose -p "$COMPOSE_PROJECT" up -d --build 2>&1 | tee "$JCI_OUTPUT_DIR/compose-up.log"
# ---- Wait for web service to respond ----
echo "=== Waiting for web service ==="
MAX_WAIT=90
ELAPSED=0
# Find the mapped port for web:8000
while [ $ELAPSED -lt $MAX_WAIT ]; do
WEB_PORT=$(docker compose -p "$COMPOSE_PROJECT" port web 8000 2>/dev/null | cut -d: -f2 || true)
if [ -n "$WEB_PORT" ]; then
# Check if web responds
if curl -sf "http://localhost:$WEB_PORT/health/" >/dev/null 2>&1; then
echo " Web service healthy on port $WEB_PORT!"
break
fi
fi
echo " [$ELAPSED s] waiting..."
sleep 5
ELAPSED=$((ELAPSED + 5))
done
if [ $ELAPSED -ge $MAX_WAIT ]; then
echo "ERROR: Services did not become healthy within ${MAX_WAIT}s"
docker compose -p "$COMPOSE_PROJECT" ps 2>&1 | tee "$JCI_OUTPUT_DIR/compose-ps.log"
docker compose -p "$COMPOSE_PROJECT" logs 2>&1 | tee "$JCI_OUTPUT_DIR/compose-logs.log"
exit 1
fi
BASE_URL="http://localhost:${WEB_PORT}"
echo "=== Web service at $BASE_URL ==="
# ---- Run API tests ----
echo "=== Running API tests ==="
bash "$PROJECT_DIR/test_api.sh" "$BASE_URL" "$JCI_OUTPUT_DIR" 2>&1 \
| tee "$JCI_OUTPUT_DIR/test-output.log"
TEST_EXIT=${PIPESTATUS[0]}
# ---- Capture logs for artifacts ----
docker compose -p "$COMPOSE_PROJECT" logs 2>&1 > "$JCI_OUTPUT_DIR/compose-logs.log"
echo "=== CI Complete (exit $TEST_EXIT) ==="
exit $TEST_EXIT
Midnight Build Telegram
$ tree .
.
├── .jci
│ ├── crontab
│ └── run.sh
└── README.md
$ cat .jci/run.sh
#!/bin/bash
set -o pipefail
cd "$JCI_REPO_ROOT" || exit 1
REPO_NAME=$(basename "$JCI_REPO_ROOT")
TIMESTAMP=$(date '+%Y-%m-%d %H:%M:%S')
SHORT_COMMIT=$(echo "$JCI_COMMIT" | head -c 7)
# ── Run Django tests ─────────────────────────────────────────
TEST_OUTPUT=$(python3 manage.py test core 2>&1)
TEST_EXIT=$?
# ── Save results to output dir ───────────────────────────────
echo "$TEST_OUTPUT" > "$JCI_OUTPUT_DIR/test_output.txt"
echo "$TEST_EXIT" > "$JCI_OUTPUT_DIR/exit_code.txt"
if [ "$TEST_EXIT" -eq 0 ]; then
STATUS="✅ PASSED"
else
STATUS="❌ FAILED"
fi
# ── Send Telegram notification ───────────────────────────────
MESSAGE=$(cat <<EOF
*Midnight Build Report*
Repo: \`${REPO_NAME}\`
Commit: \`${SHORT_COMMIT}\`
Status: ${STATUS}
Timestamp: ${TIMESTAMP}
EOF
)
if [ -n "$TELEGRAM_BOT_TOKEN" ] && [ -n "$TELEGRAM_CHAT_ID" ]; then
curl -s -X POST \
"https://api.telegram.org/bot${TELEGRAM_BOT_TOKEN}/sendMessage" \
-d chat_id="$TELEGRAM_CHAT_ID" \
-d text="$MESSAGE" \
-d parse_mode="Markdown" \
> /dev/null
else
echo "WARNING: TELEGRAM_BOT_TOKEN or TELEGRAM_CHAT_ID not set, skipping notification"
fi
exit "$TEST_EXIT"
Trufflehog Scan
$ tree .
.
├── .jci
│ ├── crontab
│ └── run.sh
└── README.md
$ cat .jci/run.sh
#!/bin/bash
set -o pipefail
cd "$JCI_REPO_ROOT" || exit 1
REPO_NAME=$(basename "$JCI_REPO_ROOT")
SHORT_COMMIT=$(echo "$JCI_COMMIT" | head -c 7)
REPORT="$JCI_OUTPUT_DIR/trufflehog-report.txt"
echo "=== TruffleHog Secret Scan ==="
echo "Repo: $REPO_NAME"
echo "Commit: $SHORT_COMMIT"
echo "Time: $(date '+%Y-%m-%d %H:%M:%S')"
echo
# ── Run trufflehog3 scan (current working tree, no history) ──
echo "Scanning current working tree..."
trufflehog3 --no-history . > "$REPORT" 2>&1 || true
echo "Scanning commit history..."
trufflehog3 --no-current . >> "$REPORT" 2>&1 || true
# ── Report findings ──────────────────────────────────────────
if [ -s "$REPORT" ]; then
FINDINGS=$(grep -c 'MEDIUM\|HIGH\|CRITICAL' "$REPORT" 2>/dev/null || echo "0")
echo "⚠️ Found $FINDINGS potential issue(s). See report:"
cat "$REPORT"
echo
echo "Report saved to trufflehog-report.txt"
# In production you might: exit 1
# For this example, we report but don't fail
else
echo "✅ No secrets found."
fi
echo "=== Scan Complete ==="
exit 0
Lint Fix Precommit
$ tree .
.
├── .jci
│ └── run.sh
├── README.md
└── install-hook.sh
$ cat .jci/run.sh
#!/bin/bash
set -euo pipefail
# Jaypore CI: Lint & fix for a multi-language repo (Python/Go/JS)
# Intended to run as a pre-commit check via `git jci run`.
#
# Environment provided by Jaypore CI:
# JCI_COMMIT - the commit being checked
# JCI_REPO_ROOT - root of the git repository
# JCI_OUTPUT_DIR - directory for build artifacts (cwd at start)
RESULTS="${JCI_OUTPUT_DIR}/lint-results.txt"
: > "$RESULTS"
cd "$JCI_REPO_ROOT"
overall_status=0
# ── Python ──────────────────────────────────────────────────────────
echo "=== Python lint ===" | tee -a "$RESULTS"
# Syntax check every .py file
py_files=$(find . -name '*.py' -not -path './.git/*' -not -path './node_modules/*' || true)
if [ -n "$py_files" ]; then
py_errors=0
while IFS= read -r f; do
if ! python3 -m py_compile "$f" 2>>"$RESULTS"; then
py_errors=$((py_errors + 1))
fi
done <<< "$py_files"
if [ "$py_errors" -gt 0 ]; then
echo "FAIL: $py_errors Python file(s) have syntax errors" | tee -a "$RESULTS"
overall_status=1
else
echo "OK: all Python files pass syntax check" | tee -a "$RESULTS"
fi
# Formatter check (black)
if command -v black &>/dev/null; then
echo "--- black --check ---" | tee -a "$RESULTS"
if ! black --check . 2>&1 | tee -a "$RESULTS"; then
echo "FAIL: black found files that need reformatting" | tee -a "$RESULTS"
echo " Run 'black .' to fix, then re-commit." | tee -a "$RESULTS"
overall_status=1
else
echo "OK: black is happy" | tee -a "$RESULTS"
fi
else
echo "SKIP: black not installed" | tee -a "$RESULTS"
fi
else
echo "SKIP: no .py files found" | tee -a "$RESULTS"
fi
# ── Go ──────────────────────────────────────────────────────────────
echo "" | tee -a "$RESULTS"
echo "=== Go lint ===" | tee -a "$RESULTS"
go_files=$(find . -name '*.go' -not -path './.git/*' -not -path './vendor/*' || true)
if [ -n "$go_files" ]; then
if command -v gofmt &>/dev/null; then
unformatted=$(gofmt -l . 2>&1 || true)
if [ -n "$unformatted" ]; then
echo "FAIL: the following Go files need formatting:" | tee -a "$RESULTS"
echo "$unformatted" | tee -a "$RESULTS"
echo " Run 'gofmt -w .' to fix, then re-commit." | tee -a "$RESULTS"
overall_status=1
else
echo "OK: all Go files are formatted" | tee -a "$RESULTS"
fi
else
echo "SKIP: gofmt not installed" | tee -a "$RESULTS"
fi
else
echo "SKIP: no .go files found" | tee -a "$RESULTS"
fi
# ── JavaScript ──────────────────────────────────────────────────────
echo "" | tee -a "$RESULTS"
echo "=== JavaScript lint ===" | tee -a "$RESULTS"
js_files=$(find . -name '*.js' -not -path './.git/*' -not -path './node_modules/*' || true)
if [ -n "$js_files" ]; then
if [ -f package.json ] && command -v npx &>/dev/null; then
echo "--- eslint --fix ---" | tee -a "$RESULTS"
# --fix rewrites files in place; the pre-commit hook should
# stage the corrections automatically.
if ! npx eslint --fix . 2>&1 | tee -a "$RESULTS"; then
echo "FAIL: eslint reported errors that could not be auto-fixed" | tee -a "$RESULTS"
overall_status=1
else
echo "OK: eslint passed (auto-fixable issues were corrected)" | tee -a "$RESULTS"
fi
else
echo "SKIP: npx/package.json not available" | tee -a "$RESULTS"
fi
else
echo "SKIP: no .js files found" | tee -a "$RESULTS"
fi
# ── Summary ─────────────────────────────────────────────────────────
echo "" | tee -a "$RESULTS"
if [ "$overall_status" -eq 0 ]; then
echo "✅ All lint checks passed." | tee -a "$RESULTS"
else
echo "❌ Some lint checks failed. See above for details." | tee -a "$RESULTS"
fi
exit "$overall_status"
Sub Pipelines
$ tree .
.
├── .jci
│ └── run.sh
├── go-app
│ ├── go-app
│ ├── go.mod
│ ├── main.go
│ └── main_test.go
├── js-app
│ ├── index.js
│ └── package.json
├── python-app
│ ├── app.py
│ └── test_app.py
└── README.md
$ cat .jci/run.sh
#!/bin/bash
set -euo pipefail
# 06-sub-pipelines: Run CI steps only for parts of the monorepo that changed.
#
# Detects which top-level folders were modified in the latest commit and
# launches the matching sub-pipeline (python / js / go). When no diff is
# available (e.g. the very first commit) every pipeline runs.
cd "$JCI_REPO_ROOT"
# ── Detect changed folders ───────────────────────────────────────────
changed_files=$(git diff --name-only HEAD~1 HEAD 2>/dev/null || true)
run_python=false
run_js=false
run_go=false
if [ -z "$changed_files" ]; then
echo "No diff detected (first commit or shallow clone) — running all sub-pipelines."
run_python=true
run_js=true
run_go=true
else
echo "Changed files:"
echo "$changed_files" | sed 's/^/ /'
echo
if echo "$changed_files" | grep -q '^06-sub-pipelines/python-app/'; then
run_python=true
fi
if echo "$changed_files" | grep -q '^06-sub-pipelines/js-app/'; then
run_js=true
fi
if echo "$changed_files" | grep -q '^06-sub-pipelines/go-app/'; then
run_go=true
fi
fi
pipelines_ran=0
summary=""
# ── Python sub-pipeline ──────────────────────────────────────────────
if $run_python; then
echo "═══ Python sub-pipeline ═══"
result_file="$JCI_OUTPUT_DIR/python-results.txt"
{
echo "Python sub-pipeline results"
echo "Commit: $JCI_COMMIT"
echo "Run at: $(date -u +%Y-%m-%dT%H:%M:%SZ)"
echo
# Lint
echo "── Lint ──"
if [ -d 06-sub-pipelines/python-app ] && command -v python3 &>/dev/null; then
if command -v ruff &>/dev/null; then
echo "Running: ruff check 06-sub-pipelines/python-app/"
ruff check 06-sub-pipelines/python-app/ 2>&1 && echo "Lint: PASS" || echo "Lint: FAIL"
elif command -v flake8 &>/dev/null; then
echo "Running: flake8 06-sub-pipelines/python-app/"
flake8 06-sub-pipelines/python-app/ 2>&1 && echo "Lint: PASS" || echo "Lint: FAIL"
else
echo "No Python linter found (ruff/flake8) — checking syntax only."
find 06-sub-pipelines/python-app -name '*.py' -exec python3 -m py_compile {} + 2>&1 \
&& echo "Syntax check: PASS" || echo "Syntax check: FAIL"
fi
else
echo "06-sub-pipelines/python-app/ directory or python3 not found — skipped."
fi
echo
# Tests
echo "── Tests ──"
if [ -d 06-sub-pipelines/python-app ] && command -v python3 &>/dev/null; then
if [ -f 06-sub-pipelines/python-app/requirements.txt ]; then
echo "Installing dependencies…"
pip install -q -r 06-sub-pipelines/python-app/requirements.txt 2>&1 || true
fi
echo "Running: python3 -m pytest 06-sub-pipelines/python-app/"
python3 -m pytest 06-sub-pipelines/python-app/ 2>&1 && echo "Tests: PASS" || echo "Tests: FAIL"
else
echo "06-sub-pipelines/python-app/ directory or python3 not found — skipped."
fi
} | tee "$result_file"
pipelines_ran=$((pipelines_ran + 1))
summary="${summary} ✔ python-app\n"
echo
fi
# ── JS sub-pipeline ──────────────────────────────────────────────────
if $run_js; then
echo "═══ JS sub-pipeline ═══"
result_file="$JCI_OUTPUT_DIR/js-results.txt"
{
echo "JS sub-pipeline results"
echo "Commit: $JCI_COMMIT"
echo "Run at: $(date -u +%Y-%m-%dT%H:%M:%SZ)"
echo
# Lint
echo "── Lint ──"
if [ -d 06-sub-pipelines/js-app ]; then
if [ -f 06-sub-pipelines/js-app/package.json ] && command -v npm &>/dev/null; then
echo "Installing dependencies…"
(cd 06-sub-pipelines/js-app && npm ci --ignore-scripts 2>&1) || true
if [ -x 06-sub-pipelines/js-app/node_modules/.bin/eslint ]; then
echo "Running: eslint 06-sub-pipelines/js-app/"
06-sub-pipelines/js-app/node_modules/.bin/eslint 06-sub-pipelines/js-app/ 2>&1 \
&& echo "Lint: PASS" || echo "Lint: FAIL"
else
echo "eslint not installed — listing JS files instead."
find 06-sub-pipelines/js-app -name '*.js' -o -name '*.ts' | head -20
echo "Lint: SKIPPED"
fi
else
echo "No package.json or npm not available — checking syntax with node."
if command -v node &>/dev/null; then
find 06-sub-pipelines/js-app -name '*.js' -exec node --check {} \; 2>&1 \
&& echo "Syntax check: PASS" || echo "Syntax check: FAIL"
else
echo "node not found — skipped."
fi
fi
else
echo "06-sub-pipelines/js-app/ directory not found — skipped."
fi
} | tee "$result_file"
pipelines_ran=$((pipelines_ran + 1))
summary="${summary} ✔ js-app\n"
echo
fi
# ── Go sub-pipeline ──────────────────────────────────────────────────
if $run_go; then
echo "═══ Go sub-pipeline ═══"
result_file="$JCI_OUTPUT_DIR/go-results.txt"
{
echo "Go sub-pipeline results"
echo "Commit: $JCI_COMMIT"
echo "Run at: $(date -u +%Y-%m-%dT%H:%M:%SZ)"
echo
# Build & test
echo "── Build & Test ──"
if [ -d 06-sub-pipelines/go-app ]; then
if command -v go &>/dev/null; then
echo "Running: go build ./06-sub-pipelines/go-app/..."
(cd 06-sub-pipelines/go-app && go build ./...) 2>&1 && echo "Build: PASS" || echo "Build: FAIL"
echo
echo "Running: go test ./06-sub-pipelines/go-app/..."
(cd 06-sub-pipelines/go-app && go test ./...) 2>&1 && echo "Test: PASS" || echo "Test: FAIL"
else
echo "go not found — skipped."
fi
else
echo "06-sub-pipelines/go-app/ directory not found — skipped."
fi
} | tee "$result_file"
pipelines_ran=$((pipelines_ran + 1))
summary="${summary} ✔ go-app\n"
echo
fi
# ── Summary ──────────────────────────────────────────────────────────
echo "═══════════════════════════════════════"
echo "Sub-pipeline summary ($pipelines_ran ran)"
echo "═══════════════════════════════════════"
if [ $pipelines_ran -eq 0 ]; then
echo " No sub-pipelines matched the changed files."
else
printf "$summary"
fi
echo
echo "Result files in $JCI_OUTPUT_DIR:"
ls -1 "$JCI_OUTPUT_DIR"/*.txt 2>/dev/null || echo " (none)"
Secrets Telegram
$ tree .
.
├── .jci
│ └── run.sh
├── README.md
└── secrets.example.json
$ cat .jci/run.sh
#!/bin/bash
set -o pipefail
# Jaypore CI run script
# ---------------------
# This script is executed by Jaypore CI.
#
# Available environment variables:
# JCI_COMMIT - The git commit being tested
# JCI_REPO_ROOT - Absolute path to the repository root
# JCI_OUTPUT_DIR - Directory for CI artifacts (cwd at start)
#
# This example demonstrates managing secrets with Mozilla SOPS.
# Secrets are stored encrypted in the repo and decrypted at CI time.
echo "=== Jaypore CI: Secrets + Telegram ==="
echo "Commit : $JCI_COMMIT"
echo "Repo : $JCI_REPO_ROOT"
echo "Output : $JCI_OUTPUT_DIR"
echo
cd "$JCI_REPO_ROOT" || exit 1
REPO_NAME=$(basename "$JCI_REPO_ROOT")
SHORT_COMMIT=$(echo "$JCI_COMMIT" | head -c 7)
TIMESTAMP=$(date '+%Y-%m-%d %H:%M:%S')
# ── Load secrets ─────────────────────────────────────────────
# Strategy:
# 1. If `sops` is installed and secrets.enc.json exists, decrypt it.
# 2. Otherwise fall back to plain environment variables.
load_secrets_from_sops() {
local secrets_file="secrets.enc.json"
if [ ! -f "$secrets_file" ]; then
echo "WARNING: $secrets_file not found, skipping SOPS decryption"
return 1
fi
echo "--- Decrypting secrets with SOPS ---"
local decrypted
decrypted=$(sops -d "$secrets_file" 2>&1)
if [ $? -ne 0 ]; then
echo "ERROR: sops decryption failed:"
echo "$decrypted"
return 1
fi
# Extract values from the decrypted JSON
TELEGRAM_BOT_TOKEN=$(echo "$decrypted" | python3 -c "import sys,json; print(json.load(sys.stdin)['TELEGRAM_BOT_TOKEN'])")
TELEGRAM_CHAT_ID=$(echo "$decrypted" | python3 -c "import sys,json; print(json.load(sys.stdin)['TELEGRAM_CHAT_ID'])")
export TELEGRAM_BOT_TOKEN TELEGRAM_CHAT_ID
echo "Secrets loaded from $secrets_file"
return 0
}
if command -v sops &> /dev/null; then
load_secrets_from_sops || echo "Falling back to environment variables"
else
echo "--- SOPS not installed ---"
echo "Install it to use encrypted secrets:"
echo " # Debian/Ubuntu"
echo " curl -LO https://github.com/getsops/sops/releases/download/v3.9.4/sops_3.9.4_amd64.deb"
echo " sudo dpkg -i sops_3.9.4_amd64.deb"
echo ""
echo " # macOS"
echo " brew install sops"
echo ""
echo "Falling back to environment variables"
fi
# ── Run Django tests ─────────────────────────────────────────
echo
echo "--- Running Django tests ---"
TEST_OUTPUT=$(python3 manage.py test core 2>&1)
TEST_EXIT=$?
echo "$TEST_OUTPUT"
echo "$TEST_OUTPUT" > "$JCI_OUTPUT_DIR/test_output.txt"
echo "$TEST_EXIT" > "$JCI_OUTPUT_DIR/exit_code.txt"
if [ "$TEST_EXIT" -eq 0 ]; then
STATUS="✅ PASSED"
else
STATUS="❌ FAILED"
fi
# ── Send Telegram notification ───────────────────────────────
MESSAGE=$(cat <<EOF
*CI Build — Secrets Example*
Repo: \`${REPO_NAME}\`
Commit: \`${SHORT_COMMIT}\`
Status: ${STATUS}
Time: ${TIMESTAMP}
EOF
)
if [ -n "$TELEGRAM_BOT_TOKEN" ] && [ -n "$TELEGRAM_CHAT_ID" ]; then
echo
echo "--- Sending Telegram notification ---"
RESPONSE=$(curl -s -X POST \
"https://api.telegram.org/bot${TELEGRAM_BOT_TOKEN}/sendMessage" \
-d chat_id="$TELEGRAM_CHAT_ID" \
-d text="$MESSAGE" \
-d parse_mode="Markdown")
echo "$RESPONSE" > "$JCI_OUTPUT_DIR/telegram_response.json"
echo "Notification sent."
else
echo
echo "WARNING: TELEGRAM_BOT_TOKEN or TELEGRAM_CHAT_ID not set, skipping notification"
fi
# ── Summary ──────────────────────────────────────────────────
echo
echo "=== Summary ==="
echo "Test result : $STATUS"
echo "Test output : test_output.txt"
echo "Exit code : exit_code.txt"
echo "All artifacts in $JCI_OUTPUT_DIR"
exit "$TEST_EXIT"
Mail On Failure
$ tree .
.
├── .jci
│ ├── crontab
│ └── run.sh
└── README.md
$ cat .jci/run.sh
#!/bin/bash
set -o pipefail
# --- Send mail via Python smtplib ---
send_mail() {
local subject="$1"
local body="$2"
python3 - "$subject" "$body" <<'PYMAIL'
import sys, smtplib, os
from email.mime.text import MIMEText
subject = sys.argv[1]
body = sys.argv[2]
msg = MIMEText(body)
msg["Subject"] = subject
msg["From"] = os.environ["MAIL_FROM"]
msg["To"] = os.environ["MAIL_TO"]
with smtplib.SMTP(os.environ["SMTP_HOST"], int(os.environ["SMTP_PORT"])) as srv:
srv.starttls()
srv.login(os.environ["SMTP_USER"], os.environ["SMTP_PASS"])
srv.sendmail(msg["From"], [msg["To"]], msg.as_string())
print("Mail sent.")
PYMAIL
}
# --- Run tests ---
cd "$JCI_REPO_ROOT"
repo_name=$(basename "$JCI_REPO_ROOT")
timestamp=$(date -u +"%Y-%m-%d %H:%M:%S UTC")
python3 manage.py test core 2>&1 | tee "$JCI_OUTPUT_DIR/test-output.txt"
exit_code=${PIPESTATUS[0]}
# --- Notify on failure ---
if [ "$exit_code" -ne 0 ]; then
subject="CI failure: ${repo_name} @ ${JCI_COMMIT:0:8}"
body="Repository : ${repo_name}
Commit : ${JCI_COMMIT}
Timestamp : ${timestamp}
Exit code : ${exit_code}
--- Test output (last 80 lines) ---
$(tail -n 80 "$JCI_OUTPUT_DIR/test-output.txt")"
if [ -n "${SMTP_HOST:-}" ] && [ -n "${MAIL_FROM:-}" ] && [ -n "${MAIL_TO:-}" ]; then
send_mail "$subject" "$body"
else
echo "WARNING: SMTP_HOST, MAIL_FROM, or MAIL_TO not set — skipping email notification"
fi
fi
exit "$exit_code"
Auto Update Deps
$ tree .
.
├── .jci
│ ├── crontab
│ └── run.sh
└── README.md
$ cat .jci/run.sh
#!/bin/bash
set -euo pipefail
# ── Auto-Update Dependencies ─────────────────────────────────────────
# Scheduled at midnight via .jci/crontab.
# Updates all pip packages, runs the test suite, and reports results.
# ─────────────────────────────────────────────────────────────────────
cd "$JCI_REPO_ROOT"
echo "==> Snapshot current dependency versions"
pip3 freeze 2>/dev/null > "$JCI_OUTPUT_DIR/before-update.txt"
echo "==> Upgrading packages from requirements.txt"
pip3 install --break-system-packages --upgrade -r requirements.txt 2>&1 | tee "$JCI_OUTPUT_DIR/upgrade-log.txt" || true
echo "==> Snapshot new dependency versions"
pip3 freeze 2>/dev/null > "$JCI_OUTPUT_DIR/after-update.txt"
echo "==> Dependency diff"
if diff "$JCI_OUTPUT_DIR/before-update.txt" "$JCI_OUTPUT_DIR/after-update.txt" \
> "$JCI_OUTPUT_DIR/dep-diff.txt" 2>&1; then
echo "No dependency changes."
else
echo "Changed packages:"
cat "$JCI_OUTPUT_DIR/dep-diff.txt"
fi
echo "==> Running test suite"
if python3 manage.py test core --no-input 2>&1 | tee "$JCI_OUTPUT_DIR/test-results.txt"; then
echo "==> Tests passed after update"
# Freeze the verified versions back into requirements.txt
# In a real setup, you'd commit updated requirements:
# pip3 freeze > requirements.txt && git add requirements.txt && git commit -m "chore: auto-update deps"
echo "Dependencies verified — would commit in a real workflow"
echo "SUCCESS" > "$JCI_OUTPUT_DIR/status.txt"
else
echo "==> Tests FAILED after update"
echo "The following packages were updated:" > "$JCI_OUTPUT_DIR/failure-report.txt"
cat "$JCI_OUTPUT_DIR/dep-diff.txt" >> "$JCI_OUTPUT_DIR/failure-report.txt"
echo "" >> "$JCI_OUTPUT_DIR/failure-report.txt"
echo "Test output:" >> "$JCI_OUTPUT_DIR/failure-report.txt"
cat "$JCI_OUTPUT_DIR/test-results.txt" >> "$JCI_OUTPUT_DIR/failure-report.txt"
echo "Failure report:"
cat "$JCI_OUTPUT_DIR/failure-report.txt"
echo "FAILURE" > "$JCI_OUTPUT_DIR/status.txt"
exit 1
fi
Build Publish Docker
$ tree .
.
├── .jci
│ └── run.sh
├── Dockerfile
└── README.md
$ cat .jci/run.sh
#!/bin/bash
set -euo pipefail
# Jaypore CI run script
# ---------------------
# This script is executed by Jaypore CI.
#
# Available environment variables:
# JCI_COMMIT - The git commit being tested
# JCI_REPO_ROOT - Absolute path to the repository root
# JCI_OUTPUT_DIR - Directory for CI artifacts (cwd at start)
#
# Optional environment variables:
# DOCKER_REGISTRY - Registry to push to (e.g. registry.example.com)
# DOCKER_USERNAME - Registry login username
# DOCKER_PASSWORD - Registry login password
#
# Any files written to JCI_OUTPUT_DIR become CI artifacts.
echo "=== Jaypore CI: Build & Publish Docker Image ==="
echo "Commit : $JCI_COMMIT"
echo "Repo : $JCI_REPO_ROOT"
echo "Output : $JCI_OUTPUT_DIR"
echo
cd "$JCI_REPO_ROOT"
IMAGE_NAME="mysite"
SHORT_SHA=$(echo "$JCI_COMMIT" | cut -c1-12)
TAG_COMMIT="${IMAGE_NAME}:${SHORT_SHA}"
TAG_LATEST="${IMAGE_NAME}:latest"
# If a registry is configured, prefix the image name
if [ -n "${DOCKER_REGISTRY:-}" ]; then
TAG_COMMIT="${DOCKER_REGISTRY}/${TAG_COMMIT}"
TAG_LATEST="${DOCKER_REGISTRY}/${TAG_LATEST}"
fi
echo "Image tags:"
echo " $TAG_COMMIT"
echo " $TAG_LATEST"
echo
# ---- 1. Build the Docker image ----
echo "--- Building Docker image ---"
docker build \
-f 10-build-publish-docker/Dockerfile \
-t "$TAG_COMMIT" \
-t "$TAG_LATEST" \
. \
2>&1 | tee "$JCI_OUTPUT_DIR/docker-build.log"
echo
echo "Build complete."
echo
# ---- 2. Run tests inside the container ----
echo "--- Running tests inside the container ---"
docker run --rm "$TAG_COMMIT" \
python3 manage.py test core --verbosity=2 \
2>&1 | tee "$JCI_OUTPUT_DIR/docker-test.log"
echo
echo "Tests passed."
echo
# ---- 3. Save image metadata ----
echo "--- Saving image info ---"
docker inspect "$TAG_COMMIT" > "$JCI_OUTPUT_DIR/image-inspect.json"
docker images --filter "reference=${IMAGE_NAME}" \
--format 'table {{.Repository}}\t{{.Tag}}\t{{.ID}}\t{{.Size}}' \
| tee "$JCI_OUTPUT_DIR/image-info.txt"
echo
# ---- 4. Push to registry (if configured) ----
if [ -n "${DOCKER_REGISTRY:-}" ]; then
echo "--- Pushing to $DOCKER_REGISTRY ---"
if [ -n "${DOCKER_USERNAME:-}" ] && [ -n "${DOCKER_PASSWORD:-}" ]; then
echo "$DOCKER_PASSWORD" | docker login "$DOCKER_REGISTRY" \
-u "$DOCKER_USERNAME" --password-stdin \
2>&1 | tee -a "$JCI_OUTPUT_DIR/docker-push.log"
fi
docker push "$TAG_COMMIT" 2>&1 | tee -a "$JCI_OUTPUT_DIR/docker-push.log"
docker push "$TAG_LATEST" 2>&1 | tee -a "$JCI_OUTPUT_DIR/docker-push.log"
echo
echo "Push complete."
else
echo "--- DOCKER_REGISTRY not set, skipping push ---"
fi
echo
# ---- 5. Summary ----
echo "=== Summary ==="
echo "Image : $TAG_COMMIT"
echo "Build log : docker-build.log"
echo "Test log : docker-test.log"
echo "Image metadata : image-inspect.json"
echo "Image info : image-info.txt"
[ -n "${DOCKER_REGISTRY:-}" ] && echo "Push log : docker-push.log"
echo "All artifacts are in $JCI_OUTPUT_DIR"
Upstream Trigger
$ tree .
.
└── README.md
Downstream Trigger
$ tree .
.
└── README.md
Jekyll Netlify
$ tree .
.
├── .jci
│ └── run.sh
├── site
│ ├── _layouts
│ │ └── default.html
│ ├── _config.yml
│ └── index.md
└── README.md
$ cat .jci/run.sh
#!/bin/bash
# ------------------------------------------------------------------
# Jaypore CI — Build Jekyll site and publish to Netlify
# ------------------------------------------------------------------
set -euo pipefail
LOG="${JCI_OUTPUT_DIR}/build.log"
exec > >(tee -a "$LOG") 2>&1
echo "=== Jekyll + Netlify CI ==="
echo "Commit : ${JCI_COMMIT:-unknown}"
echo "Repo : ${JCI_REPO_ROOT}"
echo "Output : ${JCI_OUTPUT_DIR}"
echo
# ── 1. Navigate to site source ────────────────────────────────────
SITE_DIR="${JCI_REPO_ROOT}/13-jekyll-netlify/site"
cd "$SITE_DIR"
echo "Working directory: $(pwd)"
# ── 2. Ensure Jekyll is available ─────────────────────────────────
if ! command -v jekyll &>/dev/null; then
echo "Jekyll not found — installing…"
sudo gem install jekyll bundler --no-document 2>&1 || true
else
echo "Jekyll found: $(jekyll --version)"
fi
# ── 3. Build ──────────────────────────────────────────────────────
DEST="${JCI_OUTPUT_DIR}/_site"
echo
echo "Building site → $DEST"
jekyll build --destination "$DEST"
if [ ! -d "$DEST" ]; then
echo "ERROR: Build failed — $DEST does not exist."
exit 1
fi
echo
echo "Build succeeded. Output files:"
find "$DEST" -type f | sort
echo
# ── 4. Deploy to Netlify (optional) ──────────────────────────────
if [ -n "${NETLIFY_AUTH_TOKEN:-}" ] && [ -n "${NETLIFY_SITE_ID:-}" ]; then
echo "Deploying to Netlify (site ${NETLIFY_SITE_ID})…"
ZIP_FILE="${JCI_OUTPUT_DIR}/_site.zip"
(cd "$DEST" && zip -r "$ZIP_FILE" .)
HTTP_CODE=$(curl -s -o "${JCI_OUTPUT_DIR}/netlify_response.json" \
-w "%{http_code}" \
-H "Content-Type: application/zip" \
-H "Authorization: Bearer ${NETLIFY_AUTH_TOKEN}" \
--data-binary @"$ZIP_FILE" \
"https://api.netlify.com/api/v1/sites/${NETLIFY_SITE_ID}/deploys")
if [ "$HTTP_CODE" -ge 200 ] && [ "$HTTP_CODE" -lt 300 ]; then
echo "Netlify deploy succeeded (HTTP ${HTTP_CODE})."
else
echo "WARNING: Netlify deploy returned HTTP ${HTTP_CODE}."
cat "${JCI_OUTPUT_DIR}/netlify_response.json"
exit 1
fi
else
echo "Skipping Netlify deploy: NETLIFY_AUTH_TOKEN or NETLIFY_SITE_ID not set"
fi
# ── 5. Summary ───────────────────────────────────────────────────
echo
echo "=== Summary ==="
echo "Site source : $SITE_DIR"
echo "Build output: $DEST"
echo "Build log : $LOG"
echo "Files built : $(find "$DEST" -type f | wc -l)"
echo "Done."
Docusaurus S3
$ tree .
.
├── .jci
│ └── run.sh
├── docs
│ └── index.md
├── README.md
└── build.sh
$ cat .jci/run.sh
#!/bin/bash
# Jaypore CI — Build Docusaurus (or simple static site) and publish to S3.
#
# Environment (provided by Jaypore CI):
# JCI_COMMIT — current commit SHA
# JCI_REPO_ROOT — repository root path
# JCI_OUTPUT_DIR — directory for CI artifacts (cwd at start)
#
# Optional environment:
# AWS_ACCESS_KEY_ID — AWS credentials for S3 deploy
# AWS_SECRET_ACCESS_KEY — AWS credentials for S3 deploy
# S3_BUCKET — target bucket (e.g. s3://my-docs-bucket)
# AWS_REGION — AWS region (default: us-east-1)
set -euo pipefail
PROJECT_DIR="$JCI_REPO_ROOT/14-docusaurus-s3"
BUILD_LOG="$JCI_OUTPUT_DIR/build.log"
# ── Helpers ──────────────────────────────────────────────────────────
log() { echo "[jci] $*" | tee -a "$BUILD_LOG"; }
# Start the build log
echo "=== Build log — $(date -u '+%Y-%m-%dT%H:%M:%SZ') ===" > "$BUILD_LOG"
log "Commit : ${JCI_COMMIT:-unknown}"
log "Project: $PROJECT_DIR"
log ""
cd "$PROJECT_DIR"
# ── 1. Build ─────────────────────────────────────────────────────────
if [ -f package.json ] && grep -q '"docusaurus"' package.json 2>/dev/null; then
log "Detected Docusaurus project — running npm build"
npm ci 2>&1 | tee -a "$BUILD_LOG"
npm run build 2>&1 | tee -a "$BUILD_LOG"
SITE_DIR="$PROJECT_DIR/build"
else
log "No Docusaurus project found — using simple build.sh fallback"
bash build.sh 2>&1 | tee -a "$BUILD_LOG"
SITE_DIR="$PROJECT_DIR/build"
fi
# ── 2. Copy build output to CI artifacts ─────────────────────────────
log ""
log "Copying build output to \$JCI_OUTPUT_DIR/build/"
mkdir -p "$JCI_OUTPUT_DIR/build"
cp -r "$SITE_DIR"/* "$JCI_OUTPUT_DIR/build/"
log "Artifact files:"
ls -lR "$JCI_OUTPUT_DIR/build/" 2>&1 | tee -a "$BUILD_LOG"
# ── 3. Deploy to S3 (optional) ───────────────────────────────────────
log ""
if [ -n "${AWS_ACCESS_KEY_ID:-}" ] && [ -n "${S3_BUCKET:-}" ]; then
AWS_REGION="${AWS_REGION:-us-east-1}"
log "Deploying to $S3_BUCKET (region: $AWS_REGION)"
aws s3 sync "$JCI_OUTPUT_DIR/build/" "$S3_BUCKET" \
--region "$AWS_REGION" \
--delete \
2>&1 | tee -a "$BUILD_LOG"
log "S3 deploy complete."
else
log "Skipping S3 deploy: AWS credentials or S3_BUCKET not set"
fi
# ── 4. Summary ───────────────────────────────────────────────────────
log ""
log "========================================"
log " Build & deploy finished successfully"
log " Pages : $(find "$JCI_OUTPUT_DIR/build" -name '*.html' | wc -l)"
log " Log : \$JCI_OUTPUT_DIR/build.log"
log "========================================"