fusero-app-boilerplate/README.md

# ⚡️ Fusero App Boilerplate

A full-stack application boilerplate with a React frontend and Node.js backend — powered by Fastify, Vite, PostgreSQL, Docker, and optional Kubernetes & Helm support. Built for modern dev workflows and AI-powered backend endpoint generation.

---

## 📚 Table of Contents

- [⚡️ Fusero App Boilerplate](#️-fusero-app-boilerplate)
  - [📚 Table of Contents](#-table-of-contents)
  - [📁 Project Structure](#-project-structure)
  - [⚙️ Prerequisites](#️-prerequisites)
  - [💻 Development Setup](#-development-setup)
- [To create a new migration:](#to-create-a-new-migration)
- [npm run migration:create](#npm-run-migrationcreate)
- [To apply migrations:](#to-apply-migrations)
- [To seed the database:](#to-seed-the-database)
    - [Alternate: Running Services in Separate Terminals](#alternate-running-services-in-separate-terminals)
  - [🛠️ Environment Setup](#️-environment-setup)
- [For Kubernetes, these are set in chart/values.yaml:](#for-kubernetes-these-are-set-in-chartvaluesyaml)
- [POSTGRES\_NAME=fusero-boilerplate-db](#postgres_namefusero-boilerplate-db)
- [POSTGRES\_HOSTNAME=postgres-service](#postgres_hostnamepostgres-service)
- [POSTGRES\_PORT=19095](#postgres_port19095)
- [POSTGRES\_USER=root](#postgres_userroot)
- [POSTGRES\_PASSWORD=root123](#postgres_passwordroot123)
  - [🐳 Docker Development](#-docker-development)
- [To create a new migration:](#to-create-a-new-migration-1)
- [npm run migration:create](#npm-run-migrationcreate-1)
- [To apply migrations:](#to-apply-migrations-1)
- [To seed the database:](#to-seed-the-database-1)
  - [🚀 Kubernetes Deployment](#-kubernetes-deployment)
  - [🌐 Frontend Routing in Production](#-frontend-routing-in-production)
  - [🔐 HTTPS with Self-Signed Certificates](#-https-with-self-signed-certificates)
  - [🧠 Development Best Practices](#-development-best-practices)
  - [📘 API Documentation](#-api-documentation)
  - [🧩 ChatGPT-Powered Endpoint Creation](#-chatgpt-powered-endpoint-creation)
  - [🧪 Troubleshooting](#-troubleshooting)
  - [🤝 Contributing](#-contributing)
  - [📄 License](#-license)
  - [Kubernetes Troubleshooting \& Redeployment Commands](#kubernetes-troubleshooting--redeployment-commands)
    - [1. Rebuild the backend Docker image (after code/config changes)](#1-rebuild-the-backend-docker-image-after-codeconfig-changes)
    - [2. (If using a remote registry) Push the image](#2-if-using-a-remote-registry-push-the-image)
    - [3. Upgrade the Helm release with the latest values](#3-upgrade-the-helm-release-with-the-latest-values)
    - [4. Restart the backend deployment to pick up new images and env vars](#4-restart-the-backend-deployment-to-pick-up-new-images-and-env-vars)
    - [5. Check backend pod environment variables](#5-check-backend-pod-environment-variables)
    - [6. Check backend pod logs for errors](#6-check-backend-pod-logs-for-errors)
    - [7. If you change DB env vars or code, repeat steps 1-6](#7-if-you-change-db-env-vars-or-code-repeat-steps-1-6)
  - [Frontend Rebuild \& Redeploy (Kubernetes)](#frontend-rebuild--redeploy-kubernetes)
    - [1. Rebuild the frontend Docker image](#1-rebuild-the-frontend-docker-image)
    - [2. (If using a remote registry) Push the image](#2-if-using-a-remote-registry-push-the-image-1)
    - [3. Upgrade the Helm release](#3-upgrade-the-helm-release)
    - [4. Restart the frontend deployment](#4-restart-the-frontend-deployment)
  - [Port-Forwarding for Local Access](#port-forwarding-for-local-access)
    - [Frontend (React app)](#frontend-react-app)
    - [Backend (API)](#backend-api)
    - [Database](#database)
  - [NGINX Backend Service Name: Docker Compose vs Kubernetes](#nginx-backend-service-name-docker-compose-vs-kubernetes)
    - [How to update the NGINX config for Kubernetes](#how-to-update-the-nginx-config-for-kubernetes)
  - [Cleaning Up Duplicate or Crashing Deployments and Pods in Kubernetes](#cleaning-up-duplicate-or-crashing-deployments-and-pods-in-kubernetes)
    - [1. List deployments and pods](#1-list-deployments-and-pods)
    - [2. Delete old or crashing deployments (example IDs from your cluster)](#2-delete-old-or-crashing-deployments-example-ids-from-your-cluster)
    - [3. Delete old or crashing pods (example IDs from your cluster)](#3-delete-old-or-crashing-pods-example-ids-from-your-cluster)
  - [Debugging Frontend Pod Crashes: NGINX SSL Certificate Errors](#debugging-frontend-pod-crashes-nginx-ssl-certificate-errors)
    - [How to fix for Kubernetes (Recommended)](#how-to-fix-for-kubernetes-recommended)
  - [Connecting to the Database from Your Host (DBeaver, etc.)](#connecting-to-the-database-from-your-host-dbeaver-etc)
  - [🎯 Kubernetes Namespace Management](#-kubernetes-namespace-management)
    - [Development Namespace Setup](#development-namespace-setup)
    - [Production Namespace Setup](#production-namespace-setup)
    - [Namespace Management Commands](#namespace-management-commands)
    - [Recommended Kubernetes GUI Tools](#recommended-kubernetes-gui-tools)
  - [🆕 Namespaced Development Environment with Helm](#-namespaced-development-environment-with-helm)
    - [What was changed:](#what-was-changed)
    - [How to use:](#how-to-use)
    - [Why use namespaces?](#why-use-namespaces)
  - [🔒 Production Security \& Best Practices](#-production-security--best-practices)
    - [Environment Variables \& Secrets](#environment-variables--secrets)
    - [HTTPS \& Certificates](#https--certificates)
    - [CORS \& Security Headers](#cors--security-headers)
    - [Logging \& Monitoring](#logging--monitoring)
    - [Database Backup](#database-backup)
    - [CI/CD \& Automated Testing](#cicd--automated-testing)
    - [Troubleshooting Production](#troubleshooting-production)
  - [🆕 Recent Improvements \& Troubleshooting](#-recent-improvements--troubleshooting)
  - [🚀 Production Deployment Pipeline (CI/CD)](#-production-deployment-pipeline-cicd)

---

## 📁 Project Structure

fusero-app-boilerplate/
├── chart/                  # Helm chart for Kubernetes
│   ├── Chart.yaml
│   ├── values.dev.yaml
│   ├── values.prod.yaml
│   └── templates/
├── config/
├── coverage/
├── dist/
├── docs/
├── frontend/               # React frontend app
│   ├── public/
│   └── src/
├── mikro-orm.config.ts
├── nginx/
├── node_modules/
├── package.json
├── package-lock.json
├── docker-compose.yaml
├── docker-compose.dev.yaml
├── .gitignore
├── .gitea-ci.yaml
├── .prettierrc.json
├── .eslintrc.json
├── architecture.excalidraw
├── src/                    # Node.js backend source
│   ├── apps/
│   ├── constants/
│   ├── database/
│   ├── middleware/
│   ├── plugins/
│   ├── shared/
│   ├── tests/
│   ├── types/
│   └── ...
├── test/
├── utils/
└── README.md

---

## ⚙️ Prerequisites

- Node.js (v20 or higher)
- npm (v9 or higher)
- Docker and Docker Compose
- Git

## Development Setup

### Important Note: Database Must Run in Docker
The PostgreSQL database must always run in Docker, regardless of your development setup choice. This ensures consistent database behavior across all environments.

To start the database:
```bash
docker build -t fusero-frontend-dev:local ./frontend
```

### 2. (If using a remote registry) Push the image
```bash
docker push <your-registry>/fusero-frontend-dev:local
```

### 3. Upgrade the Helm release
```bash
helm upgrade fusero ./chart -n fusero -f chart/values.dev.yaml
```

### 4. Restart the frontend deployment
```bash
kubectl rollout restart deployment/fusero-frontend -n fusero
```

---

## Port-Forwarding for Local Access

To access your services running in Kubernetes from your local machine, use these commands:

### Frontend (React app)
```bash
kubectl port-forward -n fusero-dev svc/fusero-frontend-service 3000:80
```
- Access at: http://localhost:3000

### Backend (API)
```bash
kubectl port-forward -n fusero-dev svc/fusero-backend-service 14000:14000
```
- Access at: http://localhost:14000

### Database
```bash
kubectl port-forward -n fusero-dev svc/postgres-service 5432:5432
```
- Access at: localhost:5432

---

## NGINX Backend Service Name: Docker Compose vs Kubernetes

**If your frontend uses NGINX to proxy API requests, you must update the backend service name depending on your environment:**

- **Docker Compose/local:** The backend may be named `fusero-app-backend`.
- **Kubernetes:** The backend service is named `fusero-backend-service`.

### How to update the NGINX config for Kubernetes

Edit `frontend/nginx.conf`:

**Change this:**
```nginx
proxy_pass http://fusero-app-backend:14000/;
```
**To this:**
```nginx
proxy_pass http://fusero-backend-service:14000/;
```

Then rebuild and redeploy the frontend:
```bash
docker build -t fusero-frontend-dev:local ./frontend
# (push if needed)
helm upgrade fusero ./chart -n fusero -f chart/values.dev.yaml
kubectl rollout restart deployment/fusero-frontend -n fusero
```

**If you see an NGINX error like `host not found in upstream`, this is the cause!**

---

## Cleaning Up Duplicate or Crashing Deployments and Pods in Kubernetes

If you see multiple frontend or backend pods (or CrashLoopBackOff errors), clean up your namespace with these steps:

### 1. List deployments and pods
```bash
kubectl get deployments -n fusero
kubectl get pods -n fusero
```

### 2. Delete old or crashing deployments (example IDs from your cluster)
```bash
kubectl delete deployment fusero-frontend-65cb8db99d -n fusero
kubectl delete deployment fusero-frontend-74fcbb778 -n fusero
```

### 3. Delete old or crashing pods (example IDs from your cluster)
```bash
kubectl delete pod fusero-frontend-65cb8db99d-f2lhr -n fusero
kubectl delete pod fusero-frontend-74fcbb778-v89gm -n fusero
```

**Tip:** Only keep the latest, healthy pods and deployments. If in doubt, check with `kubectl get deployments -n fusero` and `kubectl get pods -n fusero` before deleting.

---

## Debugging Frontend Pod Crashes: NGINX SSL Certificate Errors

If your frontend pod crashes with an error like:

```
nginx: [emerg] cannot load certificate "/etc/nginx/certs/fusero-selfsigned.crt": BIO_new_file() failed (SSL: error:80000002:system library::No such file or directory)
```

This means NGINX is trying to load an SSL certificate that does not exist in the pod.

### How to fix for Kubernetes (Recommended)
1. Edit `frontend/nginx.conf`:
   - Change:
     ```nginx
     listen 14443 ssl;
     ssl_certificate /etc/nginx/certs/fusero-selfsigned.crt;
     ssl_certificate_key /etc/nginx/certs/fusero-selfsigned.key;
     ```
   - To:
     ```nginx
     listen 8080;
     # (remove the ssl_certificate and ssl_certificate_key lines)
     ```
2. Rebuild the frontend Docker image:
   ```bash
   docker build --no-cache -t fusero-frontend-dev:local ./frontend
   ```
3. (If using a remote registry) Push the image.
4. Redeploy with Helm:
   ```bash
   helm upgrade fusero ./chart -n fusero -f chart/values.dev.yaml
   ```
5. Check pod status:
   ```bash
   kubectl get pods -n fusero
   ```

**This will make NGINX listen on port 8080 without SSL, which is standard for in-cluster Kubernetes services.**

---

## Connecting to the Database from Your Host (DBeaver, etc.)

To connect to the Postgres database running in Kubernetes from your local machine (for example, using DBeaver or another SQL client):

1. **Port-forward the Postgres service:**
   ```bash
   kubectl port-forward svc/postgres-service 5432:5432
   ```
   - Keep this terminal open while you use your database client.
   - If port 5432 is in use on your machine, you can use another local port (e.g., `15432:5432`) and connect to port 15432 in your client.

2. **Database connection settings:**
   - **Host:** `localhost`
   - **Port:** `5432` (or your chosen local port)
   - **Database:** `fusero-boilerplate-db`
   - **Username:** `root`
   - **Password:** `root123`

3. **Open DBeaver (or your preferred client) and create a new Postgres connection using the above settings.**

4. **Test the connection.**

---

## 🎯 Kubernetes Namespace Management

### Development Namespace Setup
```bash
# Create development namespace
kubectl create namespace fusero-dev

# Set current context to development namespace
kubectl config set-context --current --namespace=fusero-dev

# Deploy to development namespace
helm upgrade --install fusero ./chart -n fusero-dev -f chart/values.dev.yaml
```

### Production Namespace Setup
```bash
# Create production namespace
kubectl create namespace fusero-prod

# Set current context to production namespace
kubectl config set-context --current --namespace=fusero-prod

# Deploy to production namespace
helm upgrade --install fusero ./chart -n fusero-prod -f chart/values.prod.yaml
```

### Namespace Management Commands
```bash
# List all namespaces
kubectl get namespaces

# Switch between namespaces
kubectl config set-context --current --namespace=<namespace-name>

# View resources in current namespace
kubectl get all

# Delete namespace (be careful!)
kubectl delete namespace <namespace-name>
```

### Recommended Kubernetes GUI Tools
1. **Lens** - The most popular Kubernetes IDE
   - Download: https://k8slens.dev/
   - Features:
     - Real-time cluster monitoring
     - Multi-cluster management
     - Namespace isolation
     - Resource visualization
     - Log streaming
     - Terminal access

2. **K9s** - Terminal-based UI
   - Install: `brew install k9s` (Mac) or `scoop install k9s` (Windows)
   - Features:
     - Fast navigation
     - Resource management
     - Log viewing
     - Port forwarding

3. **Octant** - Web-based UI
   - Install: https://octant.dev/
   - Features:
     - Resource visualization
     - Configuration management
     - Log viewing
     - Port forwarding

---

## 🆕 Namespaced Development Environment with Helm

You have configured your Kubernetes development environment to use a dedicated namespace (`fusero-dev`) for all core services (backend, frontend, and database). This ensures resource isolation and easier management between development and production.

### What was changed:
- All Kubernetes service templates (`backend-service.yaml`, `frontend-service.yaml`, `postgres-service.yaml`) now use a namespace variable.
- The development values file (`chart/values.dev.yaml`) sets `global.namespace: fusero-dev` and updates all internal service hostnames to use this namespace.
- Helm deployments now target the `fusero-dev` namespace for all dev resources.

### How to use:
1. **Create the development namespace (if not already created):**
   ```bash
   kubectl create namespace fusero-dev
   ```

2. **Deploy all services to the namespace:**
   ```bash
   helm upgrade --install fusero ./chart -n fusero-dev -f chart/values.dev.yaml
   ```

3. **Check running pods and services:**
   ```bash
   kubectl get all -n fusero-dev
   ```

4. **If you update environment variables or service hostnames, repeat the Helm upgrade command.**

5. **If you see errors about immutable fields (e.g., for Jobs), delete the old Job before redeploying:**
   ```bash
   kubectl delete job <job-name> -n fusero-dev
   helm upgrade --install fusero ./chart -n fusero-dev -f chart/values.dev.yaml
   ```

### Why use namespaces?
- Keeps dev and prod resources isolated
- Makes it easy to clean up or redeploy all dev resources
- Prevents accidental cross-environment access

---

## 🔒 Production Security & Best Practices

### Environment Variables & Secrets

The application uses a secure secrets management approach:

1. **Development Environment**:
   - Use `.env` files locally (gitignored)
   - Copy from `.env.example` as template

2. **Production Environment**:
   - Secrets are managed through Gitea CI/CD secrets
   - Template file: `chart/secrets.prod.template.yaml`
   - Actual secrets are generated during deployment
   - Never commit actual secrets to the repository

3. **Required Secrets**:
   - Database credentials
   - Admin user credentials
   - Security keys (encryption, JWT)
   - API keys (ChatGPT, Canvas)

4. **Secrets in CI/CD**:
   - Secrets are stored in Gitea CI/CD settings
   - Automatically injected during deployment
   - Used to generate `secrets.prod.yaml` at runtime

5. **Security Best Practices**:
   - All secrets files are gitignored
   - Template files contain placeholder values
   - Production secrets are never stored in the repository
   - Regular rotation of secrets recommended

### HTTPS & Certificates
- In production, use trusted certificates (e.g., Let's Encrypt).
- Configure NGINX to enforce HTTPS.

### CORS & Security Headers
- Lock down CORS settings in production.
- Use security headers (e.g., HSTS, CSP).

### Logging & Monitoring
- Configure logging for production (e.g., ELK stack, Datadog).
- Set up basic monitoring (e.g., Prometheus, Grafana).

### Database Backup
- Regularly backup your production database.
- Example: Use `pg_dump` or a managed backup service.

### CI/CD & Automated Testing
- Run tests before deploying to production.
- Example CI/CD workflow:
  ```yaml
  # .github/workflows/deploy.yml
  name: Deploy to Production
  on:
    push:
      branch: main
  jobs:
    deploy:
      runs-on: ubuntu-latest
      steps:
        - uses: actions/checkout@v2
        - name: Run Tests
          run: npm test
        - name: Deploy to Kubernetes
          run: helm upgrade --install fusero ./chart -n fusero-prod -f chart/values.prod.yaml
  ```

### Troubleshooting Production
- Common issues:
  - Database connection errors: Check secrets and network policies.
  - Pod crashes: Check logs with `kubectl logs <pod-name> -n fusero-prod`.
- Rollback: Use `helm rollback fusero <revision> -n fusero-prod`.

---

## 🆕 Recent Improvements & Troubleshooting

- **Consistent File Extensions:** All Kubernetes and Helm YAML files now use the `.yaml` extension for consistency.
- **Secrets Management:**
  - Development secrets are stored in `chart/secrets.dev.yaml` (gitignored).
  - Production secrets are generated by CI/CD as `chart/secrets.prod.yaml` from Gitea secrets.
  - All values files (`values.dev.yaml`, `values.prod.yaml`) reference secrets via environment variables.
- **CORS Configuration:**
  - For local development, set `CORS_ORIGIN: "http://localhost:3000"` in `chart/values.dev.yaml` to allow credentialed requests from the frontend.
  - Do **not** use `*` for CORS origin if you need credentials.
- **Kubernetes Job Immutability:**
  - If you update environment variables or secrets, you must delete the old migration/seed job before redeploying:
    ```bash
    kubectl delete job fusero-backend-db-init -n fusero-dev
    helm upgrade --install fusero ./chart -n fusero-dev -f chart/values.dev.yaml -f chart/secrets.dev.yaml
    ```
- **Port Forwarding for Local Access:**
  - Backend:  
    ```bash
    kubectl port-forward -n fusero-dev svc/fusero-backend-service 14000:14000
    ```
  - Frontend:  
    ```bash
    kubectl port-forward -n fusero-dev svc/fusero-frontend-service 3000:80
    ```
  - Database:  
    ```bash
    kubectl port-forward -n fusero-dev svc/postgres-service 5432:5432
    ```
- **Testing Login with curl:**
  ```bash
  curl -i -X POST http://localhost:14000/api/v1/app/users/login \
    -H "Content-Type: application/json" \
    -H "Origin: http://localhost:3000" \
    -d '{"username":"admin","password":"admin123"}'
  ```
- **Troubleshooting 401 Errors:**
  - If login fails after a redeploy:
    - Ensure secrets and values are in sync.
    - Re-run the seed job as above.
    - Check backend logs for authentication errors:
      ```bash
      kubectl logs -n fusero-dev -l app=fusero-backend --tail=100
      ```

---

## 🚀 Production Deployment Pipeline (CI/CD)

- On every push/merge to `main`, the Gitea CI/CD pipeline will:
  1. Build and push Docker images for backend and frontend.
  2. Generate `secrets.prod.yaml` from Gitea CI/CD secrets.
  3. **Delete the old migration/seed job** (`fusero-backend-db-init`) to ensure a fresh run.
  4. Deploy the app with Helm, which triggers the migration/seed job.
  5. **Wait for the migration/seed job to complete.**
  6. **Fail the pipeline if the job fails** (with logs for debugging).
  7. Verify the deployment.

- This ensures your database is always migrated and seeded with every deploy, and you'll know immediately if something goes wrong.

- To trigger a production deployment, just push or merge to `main`.