# Emergency Access Server

A fail-safe webserver that provides secure access to multiple decryption key parts with HTTP Basic Authentication and mandatory notification system integration. Designed for emergency scenarios where key retrieval must be monitored and logged.

## Features

- **Multiple key support**: Configure multiple decryption keys with individual authentication and notification backends
- **HTTP Basic Authentication**: Secure username/password authentication for each key and health endpoint
- **Fail-safe design**: All operations require successful notification delivery
- **Dynamic endpoint system**: Emergency key access and health monitoring with simple, memorable paths
- **dschep/ntfy integration**: Real-time notifications via multiple backends (Pushover, Pushbullet, Slack, etc.)
- **Real-time log monitoring**: All application logs automatically sent to notification backends
- **Password-based security**: PBKDF2-hashed passwords with individual credentials per key
- **Caddy reverse proxy ready**: Runs on localhost for secure proxy setup
- **Systemd integration**: Automatic startup and service management
- **Comprehensive logging**: Detailed audit trail of all operations with live notifications

## Architecture

The system consists of multiple authenticated key endpoints and a health monitoring endpoint:

1. **Emergency Key Endpoints** (e.g., `/emergency-key-backup`, `/emergency-key-master`): 
   - Each serves a specific decryption key part
   - Protected by HTTP Basic Authentication with unique credentials per key
   - Individual notification backends per key
   - Custom messages per key type
   - Fails closed if authentication or notifications fail
   - Supports unlimited number of keys

2. **Health Check Endpoint** (`/health-check`):
   - Protected by HTTP Basic Authentication with dedicated credentials
   - Verifies all system components required for emergency access
   - Tests both health and all key notification backends
   - Validates file system access to dummy and all key files
   - Ensures complete emergency system readiness for all configured keys
   - Used for regular system verification

**Log Monitoring**: All application logs (WARNING level and above by default) are automatically sent to the health backends for real-time monitoring and alerting.

The server runs on localhost:1127 by default and is designed to be accessed through a Caddy reverse proxy for security and TLS termination.

## Installation

### Quick Install

Run the automated installation script as root:

```bash
sudo ./install.sh
```

The installation script will:
- Install Python dependencies
- Create service user and directories
- Set up the systemd service
- Install key management tools
- Create example configuration files

### Key Management Tools

The system includes three key management scripts:

#### 1. `add_key.py` - Add New Keys
Add new emergency access keys to your configuration:

```bash
# Interactive key addition (recommended)
python add_key.py --interactive

# Add key programmatically
python add_key.py --key-id backup \
  --file /etc/emergency-access/backup-key.txt \
  --backends matrix_sec,email \
  --create-file

# List existing keys
python add_key.py --list

# Remove a key
python add_key.py --remove backup --remove-file
```

#### 2. `manage_keys.py` - Key File Operations
Manage key files, test access, and validate configuration:

```bash
# List all key files and their status
python manage_keys.py --list-files

# Validate all key files
python manage_keys.py --validate

# Test access to a specific key
python manage_keys.py --test-key backup --password "your_password"

# Test health endpoint
python manage_keys.py --test-health --password "health_password"

# Create/rotate key files
python manage_keys.py --rotate-key backup --key-type ssh
python manage_keys.py --create-key-file /path/to/key.txt --key-type api

# Generate key content
python manage_keys.py --generate-content ssh
```

#### 3. `generate_passwords.py` - Password Management
Generate secure passwords and update configuration:

```bash
# Generate passwords for specific keys
python generate_passwords.py --keys backup master recovery

# Interactive password setup
python generate_passwords.py --interactive

# Update configuration with new passwords
python generate_passwords.py --keys backup --update-config config.json

# Hash a specific password
python generate_passwords.py --hash-password "my_secure_password"
```

### Manual Installation

1. **Install system dependencies**:
   ```bash
   # Ubuntu/Debian
   sudo apt-get update
   sudo apt-get install python3 python3-pip python3-venv

   # RHEL/CentOS/Fedora (venv is built-in with Python 3.3+)
   sudo dnf install python3 python3-pip

   # Verify venv is available
   python3 -m venv --help
   ```

2. **Create service user**:
   ```bash
   sudo groupadd --system emergency-access
   sudo useradd --system --gid emergency-access --home-dir /opt/emergency-access \
                --shell /bin/false emergency-access
   ```

3. **Setup directories**:
   ```bash
   sudo mkdir -p /opt/emergency-access /etc/emergency-access
   sudo chown emergency-access:emergency-access /opt/emergency-access /etc/emergency-access
   sudo chmod 755 /opt/emergency-access
   sudo chmod 750 /etc/emergency-access
   ```

4. **Install application**:
   ```bash
   sudo cp *.py requirements.txt /opt/emergency-access/
   sudo cp config.json.example /etc/emergency-access/config.json
   sudo chown -R emergency-access:emergency-access /opt/emergency-access
   ```

5. **Setup Python environment**:
   ```bash
   sudo -u emergency-access python3 -m venv /opt/emergency-access/venv
   sudo -u emergency-access /opt/emergency-access/venv/bin/pip install -r /opt/emergency-access/requirements.txt
   ```

6. **Install systemd service**:
   ```bash
   sudo cp emergency-access.service /etc/systemd/system/
   sudo systemctl daemon-reload
   ```

## Configuration

### Main Configuration File

Copy and edit the example configuration to `/etc/emergency-access/config.json`:

```bash
sudo cp config.json.example /etc/emergency-access/config.json
sudo nano /etc/emergency-access/config.json
```

Configuration structure:

```json
{
  "server": {
    "host": "127.0.0.1",
    "port": 1127
  },
  "routes": {
    "health_route": "/health-check",
    "health_username": "health_monitor",
    "health_password_hash": "salt:hash_generated_by_password_tool"
  },
  "files": {
    "dummy_file": "/etc/emergency-access/dummy.txt"
  },
  "keys": {
    "backup_key": {
      "route": "/emergency-key-backup",
      "file": "/etc/emergency-access/backup-key.txt",
      "username": "emergency_backup",
      "password_hash": "salt:hash_generated_by_password_tool",
      "backends": ["matrix_sec", "pushover_emergency"],
      "message": "🚨 EMERGENCY: Backup decryption key accessed from server"
    },
    "master_key": {
      "route": "/emergency-key-master",
      "file": "/etc/emergency-access/master-key.txt",
      "username": "emergency_master",
      "password_hash": "salt:hash_generated_by_password_tool",
      "backends": ["matrix_sec", "pushover_critical", "slack_emergency"],
      "message": "🚨 CRITICAL: Master decryption key accessed from server"
    },
    "recovery_key": {
      "route": "/emergency-key-recovery",
      "file": "/etc/emergency-access/recovery-key.txt",
      "username": "emergency_recovery",
      "password_hash": "salt:hash_generated_by_password_tool",
      "backends": ["matrix_sec", "email_emergency"],
      "message": "🚨 EMERGENCY: Recovery decryption key accessed from server"
    },
    "admin_key": {
      "route": "/emergency-key-admin",
      "file": "/etc/emergency-access/admin-key.txt",
      "username": "emergency_admin",
      "password_hash": "salt:hash_generated_by_password_tool",
      "backends": [
        "matrix_sec",
        "pushover_critical",
        "slack_emergency",
        "email_critical"
      ],
      "message": "🚨 CRITICAL ALERT: Administrator master key accessed from server"
    }
  },
  "notifications": {
    "health_backends": ["matrix_health"],
    "config_path": "/etc/emergency-access/ntfy.yml",
    "health_message": "✅ Emergency access server health check completed",
    "log_level": "WARNING",
    "send_all_logs": true
  }
}
```

### Configuration Options

#### Server Settings
- `host`: Bind address (default: `127.0.0.1` for localhost only)
- `port`: Listen port (default: `1127`)

#### Route Settings
- `health_route`: Path for health checks (e.g., `/health-check`)
- `health_username`: Username for health endpoint authentication
- `health_password_hash`: PBKDF2 password hash for health endpoint

#### Keys Configuration
Each key in the `keys` object supports:
- `route`: Path for this specific key (e.g., `/emergency-key-backup`)
- `file`: Path to the key file for this specific key
- `username`: Username for this key's authentication
- `password_hash`: PBKDF2 password hash for this key
- `backends`: List of notification backend names for this specific key
- `message`: Custom message sent when this specific key is accessed

#### File Settings
- `dummy_file`: Path to dummy content for health checks

#### Notification Settings
- `health_backends`: List of backend names from `/etc/emergency-access/ntfy.yml` for health check notifications and all application logs
- `config_path`: Path to the ntfy configuration file (default: `/etc/emergency-access/ntfy.yml`)
- `health_message`: Message sent for health checks
- `log_level`: Minimum log level to send to health backends ("INFO", "WARNING", "ERROR")
- `send_all_logs`: Whether to send application logs to health backends (true/false)

#### Backend Name Examples
- `matrix_sec`: Matrix backend for security alerts
- `pushover_emergency`: Pushover backend for emergency notifications
- `slack_critical`: Slack backend for critical alerts
- Any backend name configured in `/etc/emergency-access/ntfy.yml`

### Key Files and Authentication Setup

1. **Generate secure passwords and hashes**:
   ```bash
   # Generate passwords for all configured keys
   python generate_passwords.py --keys backup master recovery admin
   
   # Interactive password setup
   python generate_passwords.py --interactive
   
   # Update existing config with new passwords
   python generate_passwords.py --keys backup master --update-config config.json
   ```

2. **Create key files for each configured key**:
   ```bash
   # Backup key
   echo "YOUR_BACKUP_KEY_PART_HERE" | sudo tee /etc/emergency-access/backup-key.txt
   sudo chown emergency-access:emergency-access /etc/emergency-access/backup-key.txt
   sudo chmod 600 /etc/emergency-access/backup-key.txt

   # Master key
   echo "YOUR_MASTER_KEY_PART_HERE" | sudo tee /etc/emergency-access/master-key.txt
   sudo chown emergency-access:emergency-access /etc/emergency-access/master-key.txt
   sudo chmod 600 /etc/emergency-access/master-key.txt

   # Recovery key
   echo "YOUR_RECOVERY_KEY_PART_HERE" | sudo tee /etc/emergency-access/recovery-key.txt
   sudo chown emergency-access:emergency-access /etc/emergency-access/recovery-key.txt
   sudo chmod 600 /etc/emergency-access/recovery-key.txt

   # Admin key
   echo "YOUR_ADMIN_KEY_PART_HERE" | sudo tee /etc/emergency-access/admin-key.txt
   sudo chown emergency-access:emergency-access /etc/emergency-access/admin-key.txt
   sudo chmod 600 /etc/emergency-access/admin-key.txt
   ```

3. **Create dummy file**:
   ```bash
   echo "system_healthy" | sudo tee /etc/emergency-access/dummy.txt
   sudo chown emergency-access:emergency-access /etc/emergency-access/dummy.txt
   sudo chmod 644 /etc/emergency-access/dummy.txt
   ```

4. **Store credentials securely**:
   - Save usernames and passwords in a secure password manager
   - Document which credentials correspond to which keys
   - Keep emergency access credentials separate from regular system credentials
   - Consider printing credentials and storing in a physical safe for true emergency scenarios

## dschep/ntfy Backend Setup

The system uses a dedicated ntfy configuration file at `/etc/emergency-access/ntfy.yml`. Configure your notification backends in this file and reference them by name in the main configuration.

### Configuring Notification Backends

1. **Edit the dedicated ntfy config** (`/etc/emergency-access/ntfy.yml`):
   ```yaml
   backends:
     - matrix_sec
     - matrix_health
     - pushover_emergency
   
   matrix_sec:
     backend: matrix
     url: https://your-matrix-server.com
     roomId: "!emergency-security:your-matrix-server.com"
     userId: "@emergency-bot:your-matrix-server.com"
     password: "your-matrix-bot-password"
   
   matrix_health:
     backend: matrix
     url: https://your-matrix-server.com
     roomId: "!emergency-health:your-matrix-server.com"
     userId: "@emergency-bot:your-matrix-server.com"
     password: "your-matrix-bot-password"
   
   pushover_emergency:
     backend: pushover
     user_key: YOUR_PUSHOVER_USER_KEY
     priority: 2
     retry: 60
     expire: 3600
     sound: siren
   ```

2. **Reference backend names** in emergency access config:
   ```json
   "notifications": {
     "key_backends": ["matrix_sec", "pushover_emergency"],
     "health_backends": ["matrix_health"],
     "config_path": "/etc/emergency-access/ntfy.yml"
   }
   ```

### Adding Additional Backends

Add more backends to `/etc/emergency-access/ntfy.yml` as needed:

```yaml
# Additional backends in /etc/emergency-access/ntfy.yml
backends:
  - matrix_sec
  - matrix_health
  - pushover_emergency
  - slack_critical
  - email_emergency

slack_critical:
  backend: slack
  token: YOUR_SLACK_BOT_TOKEN
  recipient: "#emergency-alerts"

email_emergency:
  backend: email
  smtp_server: smtp.gmail.com
  smtp_port: 587
  username: your-email@gmail.com
  password: your-app-password
  to: emergency-team@company.com
  from: emergency-access@company.com
```

## Service Management

### Start and Enable Service

```bash
# Start the service
sudo systemctl start emergency-access

# Enable automatic startup
sudo systemctl enable emergency-access

# Check status
sudo systemctl status emergency-access
```

### Monitoring and Logs

```bash
# View real-time logs
sudo journalctl -u emergency-access -f

# View log file
sudo tail -f /var/log/emergency-access.log

# Check service health (through Caddy proxy)
curl https://your-domain.com/health-check-b8e3f4a2

# Or directly to local service (for testing)
curl http://localhost:1127/health-check-b8e3f4a2
```

## Usage

### Emergency Key Access

```bash
# Access backup key with HTTP Basic Auth
curl -u emergency_backup:PASSWORD https://your-domain.com/emergency-key-backup

# Expected response:
{
  "success": true,
  "key_id": "backup_key",
  "key_part": "YOUR_BACKUP_KEY_PART_HERE",
  "timestamp": 1703123456.789,
  "notified_backends": ["matrix_sec", "pushover_emergency"]
}

# Access master key with HTTP Basic Auth
curl -u emergency_master:PASSWORD https://your-domain.com/emergency-key-master

# Expected response:
{
  "success": true,
  "key_id": "master_key",
  "key_part": "YOUR_MASTER_KEY_PART_HERE",
  "timestamp": 1703123456.789,
  "notified_backends": ["matrix_sec", "pushover_critical", "slack_emergency"]
}

# Access admin key with HTTP Basic Auth
curl -u emergency_admin:PASSWORD https://your-domain.com/emergency-key-admin

# Expected response:
{
  "success": true,
  "key_id": "admin_key",
  "key_part": "YOUR_ADMIN_KEY_PART_HERE",
  "timestamp": 1703123456.789,
  "notified_backends": ["matrix_sec", "pushover_critical", "slack_emergency", "email_critical"]
}

# Authentication failure response:
{
  "error": "Invalid credentials"
}
```

### Health Check

```bash
# Regular health monitoring with HTTP Basic Auth
curl -u health_monitor:PASSWORD https://your-domain.com/health-check

# Expected response (all systems operational):
{
  "status": "ok",
  "timestamp": 1703123456.789,
  "health_backends_notified": ["matrix_health"],
  "dummy_content_length": 14,
  "keys_accessible": 4,
  "key_files_status": {
    "backup_key": true,
    "master_key": true,
    "recovery_key": true,
    "admin_key": true
  },
  "key_backends_status": {
    "backup_key": {
      "backends": ["matrix_sec", "pushover_emergency"],
      "success": true
    },
    "master_key": {
      "backends": ["matrix_sec", "pushover_critical", "slack_emergency"],
      "success": true
    },
    "recovery_key": {
      "backends": ["matrix_sec", "email_emergency"],
      "success": true
    },
    "admin_key": {
      "backends": ["matrix_sec", "pushover_critical", "slack_emergency", "email_critical"],
      "success": true
    }
  },
  "emergency_system_ready": true
}

# Error response (when components fail):
{
  "status": "error",
  "message": "System components failed",
  "details": [
    "dummy file access failed: Permission denied",
    "key file access failed for 'master_key'",
    "key backends failed for 'recovery_key': Configuration error"
  ],
  "health_notifications": true,
  "dummy_file_access": false,
  "key_files_status": {
    "backup_key": true,
    "master_key": false,
    "recovery_key": true,
    "admin_key": true
  },
  "key_backends_status": {
    "backup_key": {
      "backends": ["matrix_sec", "pushover_emergency"],
      "success": true
    },
    "master_key": {
      "backends": ["matrix_sec", "pushover_critical"],
      "success": true
    },
    "recovery_key": {
      "backends": ["matrix_sec", "email_emergency"],
      "success": false,
      "error": "Configuration error"
    },
    "admin_key": {
      "backends": ["matrix_sec", "pushover_critical", "slack_emergency", "email_critical"],
      "success": true
    }
  }
}
```

## Security Considerations

### Caddy Reverse Proxy Setup

1. **Basic Caddyfile configuration**:
   ```caddy
   emergency.yourdomain.com {
       tls your-email@example.com
       reverse_proxy localhost:1127
       
       header {
           X-Content-Type-Options nosniff
           X-Frame-Options DENY
       }
       
       rate_limit {
           zone emergency {
               key {remote_host}
               events 10
               window 1m
           }
       }
   }
   ```

2. **Path-based routing**:
   ```caddy
   yourdomain.com {
       handle /emergency/* {
           uri strip_prefix /emergency
           reverse_proxy localhost:1127
       }
   }
   ```

3. **IP-restricted access**:
   ```caddy
   :443 {
       @allowed_ips remote_ip 192.168.0.0/16
       handle @allowed_ips {
           reverse_proxy localhost:1127
       }
       handle {
           respond "Access Denied" 403
       }
   }
   ```

### Network Security

The service binds only to localhost (127.0.0.1:1127) and is accessed through your existing Caddy reverse proxy. No additional firewall configuration is required.

### File Permissions

- Key file: `600` (owner read-only)
- Config file: `640` (owner read/write, group read)
- Application files: `644` (standard read permissions)

### Monitoring

1. **Set up regular health checks**:
   ```bash
   # Cron job for health monitoring through Caddy
   */5 * * * * curl -s https://yourdomain.com/health-check-b8e3f4a2 > /dev/null
   
   # Or direct to service for internal monitoring
   */5 * * * * curl -s http://localhost:1127/health-check-b8e3f4a2 > /dev/null
   ```

2. **Monitor notification delivery**:
   - Configure notification backends (Pushover, Pushbullet, etc.)
   - Set up monitoring of notification delivery in your backends
   - Monitor log files for errors
   - All application logs are automatically sent to health backends for real-time monitoring

## Troubleshooting

### Common Issues

1. **Service won't start**:
   ```bash
   sudo journalctl -u emergency-access -n 50
   sudo systemctl status emergency-access
   ```

2. **Notification failures**:
   ```bash
   # Test dschep/ntfy installation and configuration
   ntfy -c /etc/emergency-access/ntfy.yml send "test message"
   
   # Check ntfy configuration
   cat /etc/emergency-access/ntfy.yml
   
   # Test specific backend
   ntfy -c /etc/emergency-access/ntfy.yml -b matrix_sec send "test message"
   ```

3. **File permission errors**:
   ```bash
   sudo chown -R emergency-access:emergency-access /opt/emergency-access
   sudo chown emergency-access:emergency-access /etc/emergency-access/*
   ```



### Configuration Validation

Test your setup before deployment:

```bash
# Validate configuration
sudo -u emergency-access /opt/emergency-access/venv/bin/python /opt/emergency-access/main.py --validate

# Test notifications manually with your backend
ntfy -c /etc/emergency-access/ntfy.yml -b matrix_sec send "Test notification"
ntfy -c /etc/emergency-access/ntfy.yml -b pushover_emergency send "Test emergency notification"
```

## Development

### Running in Development Mode

```bash
# Create virtual environment
python3 -m venv venv
source venv/bin/activate

# Install dependencies
pip install -r requirements.txt

# Run with development config
EMERGENCY_CONFIG=config.json python main.py
```

### Testing

```bash
# Run comprehensive test suite
python test.py

# Run quick configuration tests only
python test.py --quick

# Generate passwords for configuration
python generate_passwords.py --keys backup master recovery

# Test specific endpoints with authentication
curl -u emergency_backup:PASSWORD http://localhost:1127/emergency-key-backup
curl -u emergency_master:PASSWORD http://localhost:1127/emergency-key-master
curl -u health_monitor:PASSWORD http://localhost:1127/health-check
```

## Multiple Key Implementation Details

### Authentication Security

#### HTTP Basic Authentication
- **Username/Password**: Each key and health endpoint has unique credentials
- **PBKDF2 Hashing**: Passwords are hashed using PBKDF2 with SHA-256 and 100,000 iterations
- **Salt Protection**: Each password hash includes a unique cryptographic salt
- **No Plaintext Storage**: Only password hashes are stored in configuration

#### Why HTTP Basic Auth?
1. **Standard Protocol**: Built into HTTP, supported by all clients
2. **Simple & Reliable**: No complex token management or infrastructure required
3. **Emergency Friendly**: Works with basic tools like curl in emergency situations
4. **Fail-Safe Compatible**: Authentication failure prevents access, maintaining security

### Dynamic Route Registration
The application dynamically registers routes for each configured key:
```python
for key_id, key_config in config.keys.items():
    handler = create_key_handler(key_config)
    endpoint_name = f'get_key_part_{key_id}'
    app.add_url_rule(key_config.route, endpoint_name, handler, methods=['GET'])
```

### Individual Key Handlers
Each key gets its own handler function with specific configuration:
```python
def create_key_handler(key_config):
    def get_key_part():
        # Send notifications to key-specific backends
        notification_success, successful_backends = send_ntfy_notification(
            key_config.backends,
            key_config.message,
            "EMERGENCY ACCESS ALERT"
        )
        # Return key with key ID
        return jsonify({
            'success': True,
            'key_id': key_config.key_id,
            'key_part': content,
            'timestamp': time.time(),
            'notified_backends': successful_backends
        })
    return get_key_part
```

### Health Check Validation
Health checks now validate all configured keys:
```python
# Test all key files
for key_id, key_config in config.keys.items():
    key_file_success, key_content = read_file_safely(key_config.file_path)
    key_files_status[key_id] = key_file_success
```

### Generate Secure Configuration

Use the included password generator for secure authentication:

```bash
# Generate passwords for specific keys
python generate_passwords.py --keys backup master recovery

# Interactive password setup
python generate_passwords.py --interactive

# Generate and update existing config
python generate_passwords.py --keys backup master --update-config config.json

# Hash a specific password
python generate_passwords.py --hash-password "my_secure_password"
```

The password generator creates:
- **Secure Passwords**: Cryptographically secure random passwords
- **PBKDF2 Hashes**: Salted password hashes for configuration
- **Configuration Updates**: Automatically updates config files with hashes
- **Credential Management**: Saves credentials securely for reference

### Security Best Practices

#### Credential Storage
- Store usernames and passwords in secure password manager
- Document which credentials correspond to which keys
- Never store passwords in plain text logs or configuration files
- Use the password generator for cryptographically secure passwords

#### Access Monitoring
- Monitor reverse proxy logs for authentication attempts
- Set up alerts for failed authentication attempts
- Set up alerts for any successful key access
- Regularly audit notification delivery
- Implement rate limiting to prevent brute force attacks

#### Emergency Procedures
1. **Credential Compromise**: Change the password immediately, update configuration
2. **Failed Authentication**: Investigate immediately, check logs for brute force attempts
3. **Password Leak**: Generate new passwords and update configuration
4. **Notification Failure**: Validate all backends before deployment

### Security Architecture

This implementation uses HTTP Basic Authentication for robust security:

#### Authentication-First Design
- All endpoints require HTTP Basic Authentication
- Each key has unique username and password
- Health endpoint has separate authentication credentials
- Password hashes use PBKDF2 with SHA-256 and strong salts
- No security through obscurity - proper authentication instead

## License

This project is designed for emergency access scenarios. Use responsibly and ensure proper security measures are in place.

## Support

For issues and questions:
1. Check the logs: `/var/log/emergency-access.log`
2. Verify configuration: `/etc/emergency-access/config.json`
3. Test notification systems independently
4. Monitor service status: `systemctl status emergency-access`
5. Use password generator: `python generate_passwords.py --help`

### Validation Commands
```bash
# Validate configuration
python -c "from config import Config; Config('/etc/emergency-access/config.json')"

# Test file access
sudo -u emergency-access cat /etc/emergency-access/backup-key.txt

# Test notifications
ntfy -c /etc/emergency-access/ntfy.yml -b matrix_sec send "Test message"

# Generate secure passwords
python generate_passwords.py --keys backup master recovery

# Test authentication
curl -u emergency_backup:PASSWORD http://localhost:1127/emergency-key-backup
```