# Backend Documentation (`express-backend.js`)

## Overview

The backend is built using Node.js and the Express.js framework. It serves as the central API for the RF NeRF Visualization System, handling user authentication, data management, job processing orchestration, and real-time communication via WebSockets.

## Core Components

1.  **Express Server:** Sets up the main HTTP server, configures middleware (CORS, Helmet, JSON parsing, logging), and defines API routes.
2.  **MongoDB (Mongoose):** Used as the primary database for storing user information, dataset metadata, and job details. Mongoose provides object data modeling (ODM).
3.  **User Authentication (JWT):** Implements user registration and login using bcrypt for password hashing and JSON Web Tokens (JWT) for session management. Middleware (`authenticateJWT`) protects routes.
4.  **Dataset Management:** Provides API endpoints for creating, retrieving, updating (uploading files), and deleting datasets. Includes middleware (`checkStorageLimits`, `checkSubscription`) to enforce user-specific constraints.
5.  **File Storage (Multer):** Handles file uploads using `multer`. Files are stored locally in a structured directory based on user ID (`STORAGE_DIR`). Includes file type filtering and size limits.
6.  **Job Processing Queue (Bull/Redis):** Uses Bull, backed by Redis, to manage a queue for asynchronous processing tasks (NeRF training, analysis, etc.). Jobs are added to the queue via API requests.
7.  **Python Process Execution:** The Bull queue worker executes the Python processing script (`python-processing.py`) as a separate process (`child_process.exec`). It passes necessary arguments like job type, dataset paths, output directory, and parameters.
8.  **WebSocket Server (ws):** Provides real-time updates to connected frontend clients about job progress, completion, or failure. Uses JWT for authenticating WebSocket connections.
9.  **Middleware:** Includes middleware for authentication (`authenticateJWT`), subscription checks (`checkSubscription`), and storage limit enforcement (`checkStorageLimits`).

## Database Models

*   **User:** Stores user credentials (hashed password), profile information (name, organization), role, subscription details (tier, expiry), API key, and storage usage.
*   **Dataset:** Stores dataset metadata (name, description, data type, owner), associated file information (filename, path, size, upload date), and public/private status.
*   **Job:** Stores job details (name, type, status, parameters), references to the user and dataset, progress, start/end times, priority (based on user subscription), and results (output file paths, visualization data, statistics, errors).

## API Endpoints (Illustrative)

*   **Authentication:**
    *   `POST /api/auth/register`: User registration.
    *   `POST /api/auth/login`: User login, returns JWT.
    *   `GET /api/auth/me`: Get current user profile.
    *   `PUT /api/auth/password`: Change user password.
*   **Datasets:**
    *   `POST /api/datasets`: Create a new dataset.
    *   `GET /api/datasets`: List user's datasets (with filtering/pagination).
    *   `GET /api/datasets/:id`: Get details of a specific dataset.
    *   `POST /api/datasets/:id/upload`: Upload files to a dataset.
    *   `DELETE /api/datasets/:id`: Delete a dataset and associated files.
*   **Jobs:**
    *   `POST /api/jobs`: Create and queue a new processing job.
    *   `GET /api/jobs`: List user's jobs (with filtering/pagination).
    *   `GET /api/jobs/:id`: Get details of a specific job.
    *   `DELETE /api/jobs/:id`: Delete a job (attempts to remove from queue if active).
*   **Results & Visualization:**
    *   `GET /api/results/:jobId/:filename`: Download a specific output file from a completed job.
    *   `GET /api/visualization/:jobId`: Get the processed visualization data (e.g., `visualization.json`) for a completed job.
*   **Subscription:**
    *   `POST /api/subscription/upgrade`: (Placeholder) Upgrade user subscription tier.

## Real-time Communication (WebSockets)

*   Clients connect to the WebSocket server, authenticating with their JWT.
*   The backend emits messages to relevant clients when:
    *   A job's progress percentage updates (`type: 'progress'`).
    *   A job successfully completes (`type: 'completed'`).
    *   A job fails (`type: 'failed'`).

### WebSocket Security Implementation

* **Connection Authentication**: WebSocket connections require a valid JWT in the connection URL or headers
* **Message Validation**: All received messages are validated for proper structure and permissions
* **User-specific Channels**: Messages are only sent to clients with proper authorization for the specific data
* **Reconnection Handling**: Includes logic for secure reconnection with automatic re-authentication
* **Rate Limiting**: Implements connection and message rate limiting to prevent DoS attacks
* **Connection Events**:
  * `connection`: Validates JWT and establishes user-specific socket mapping
  * `message`: Processes incoming messages after validation
  * `close`: Cleans up resources and handles potential reconnection
  * `error`: Logs errors and performs secure error handling

## Environment Variables

*   `PORT`: Server port (defaults to 3001).
*   `MONGODB_URI`: Connection string for MongoDB.
*   `JWT_SECRET`: Secret key for signing JWTs.
*   `REDIS_HOST`: Redis server hostname.
*   `REDIS_PORT`: Redis server port.
*   `PYTHON_PROCESSING_SCRIPT`: Path to the main Python processing script.
*   `STORAGE_DIR`: Base directory for storing uploaded datasets and results.
*   `MAX_UPLOAD_SIZE`: Maximum allowed file upload size.

## Setup & Running

1.  Ensure MongoDB and Redis are running.
2.  Install dependencies: `npm install`.
3.  Configure environment variables (e.g., in a `.env` file).
4.  Start the server: `node express-backend.js`.