Security Checks and Hardening
API?leak remediation & hardening playbook
This is tuned for your stack (Debian 12 in LXC with Docker, Apache2 reverse proxy to Uvicorn/Streamlit & Node/n8n). The leak source was /var/www/html/flast/.env
.
0) Immediate containment (do these first)
# 0.1 Rotate/revoke all keys found in the leaked .env (provider dashboards)
# Then update the container with the new values after step 2 below.
# 0.2 Make the current file unreadable by anyone except owner (as a stopgap)
chmod 600 /var/www/html/flast/.env
chown root:root /var/www/html/flast/.env # temporary; see step 2 for final layout
# 0.3 Block public access to dotfiles at the web tier (Apache)
cat >/etc/apache2/conf-available/deny-dotfiles.conf <<'EOF'
# Deny all access to dotfiles like .env, .git, .ht*, etc.
<FilesMatch "^\.">
Require all denied
</FilesMatch>
Options -Indexes
EOF
a2enconf deny-dotfiles
systemctl reload apache2
# 0.4 Quick sanity: these must NOT return the file contents
curl -sI http://127.0.0.1/.env | head -n1 # expect 403/404
curl -skI https://127.0.0.1/.env | head -n1 # if SSL locally
1) Forensics: did anyone fetch .env
?
# Access attempts to .env (status code focus)
zgrep -h "\.env" /var/log/apache2/access.log* 2>/dev/null | \
awk '{print $(NF-1)}' | sort | uniq -c | sort -nr
# Top IPs that tried
zgrep -h "\.env" /var/log/apache2/access.log* 2>/dev/null | \
awk '{print $1}' | sort | uniq -c | sort -nr | head
# Full lines for any 200/206 responses (bad)
zgrep -h "\.env" /var/log/apache2/access.log* 2>/dev/null | \
awk '$9 ~ /^(200|206)$/ {print}' | tail -n +1
# If behind reverse proxy / other vhosts, also scan their logs
If you see 200/206, assume the contents were retrieved. Rotate all affected secrets and audit for misuse.
2) Move secrets out of the webroot (recommended layout)
Put app secrets in a root?only directory outside the document root, and load via systemd or Docker secrets.
2A) If running services via systemd on the CT
# Create a dedicated dir for secrets
install -d -m 0750 -o root -g root /etc/flast
install -m 0640 -o root -g root /etc/flast/flast.env
# Paste NEW rotated values into /etc/flast/flast.env in KEY=VALUE lines
# Point services to this EnvironmentFile
mkdir -p /etc/systemd/system/flast.server.service.d /etc/systemd/system/flast.client.service.d
cat >/etc/systemd/system/flast.server.service.d/env.conf <<'EOF'
[Service]
EnvironmentFile=/etc/flast/flast.env
EOF
cat >/etc/systemd/system/flast.client.service.d/env.conf <<'EOF'
[Service]
EnvironmentFile=/etc/flast/flast.env
EOF
# Reload & restart
systemctl daemon-reload
systemctl restart flast.server.service flast.client.service
# Verify the new env is in effect (will print service env, redact if needed)
systemctl show flast.server.service | grep ^Environment=
2B) If using Docker for the app
Prefer Docker secrets or a bind?mounted file owned by root:
# docker-compose.yml (example fragment)
services:
app:
image: your/app
env_file: [] # avoid plaintext here for sensitive keys
secrets:
- flast_env
secrets:
flast_env:
file: /etc/flast/flast.env # managed on the host CT as above
Then docker compose up -d
to apply.
Remove/var/www/html/flast/.env
after services run cleanly from/etc/flast/flast.env
.
shred -u /var/www/html/flast/.env # permanently delete old copy
3) Extra web hardening
Apache: ensure no directory indexes or source exposure.
# In your site .conf (e.g., /etc/apache2/sites-available/*.conf)
<Directory /var/www/html>
Options -Indexes -Includes -ExecCGI
AllowOverride None
Require all granted
</Directory>
# (already added) protect dotfiles in conf-available/deny-dotfiles.conf
Reload:
apachectl -t && systemctl reload apache2
If you ever switch to Nginx:
location ~ /\. { deny all; }
autoindex off;
4) Process/env hygiene
After rotating keys, restart any process that might still have the old key in its environment or memory.
systemctl restart flast.server.service flast.client.service apache2 docker
# Spot-check that no process still carries sensitive env
for pid in $(pgrep -u flast -x python3 node); do
sudo tr '\0' '\n' </proc/$pid/environ | egrep -i 'KEY|TOKEN|SECRET' || true
done
5) Backups: avoid bundling plaintext secrets
Your vzdump LXC backups currently include the webroot. Options:
- Best: keep secrets in
/etc/flast/flast.env
and re-provision on restore (Ansible, script, or manual). No secrets in backups. - If you must exclude a path: add to your job or global config:
# One-off
vzdump 445 --exclude-path /var/www/html/flast/.env ...
# Or in /etc/vzdump.conf
# exclude-path: /var/www/html/flast/.env
Note: Excluding secrets means restores need a post-step to re-create /etc/flast/flast.env
.
If/when you move to Proxmox Backup Server, use client?side encryption and keep secrets off the image.
6) File system & repo checks
# Search for other env files near the webroot
find /var/www -type f \( -name '.env' -o -name '*.env' \) -maxdepth 4
# If the project uses git, ensure .env is ignored and not committed
grep -nE '^\s*\.env(\b|$)' /var/www/html/flast/.gitignore || echo 'Missing .env in .gitignore'
git -C /var/www/html/flast log -p --follow -- .env 2>/dev/null | tail -n +1 || echo 'No .env in git history (good)'
If .env
ever hit git history, purge with git filter-repo
(or git filter-branch
) and rotate keys again.
7) Optional: fail2ban rule to block .env
scanners
# /etc/fail2ban/filter.d/apache-no-dotenv.conf
[Definition]
failregex = <HOST> - - \[.*\] "(GET|POST) /.*\.env.* HTTP/.*" 4(?:0[34]|5\d)
ignoreregex =
# /etc/fail2ban/jail.d/apache-no-dotenv.local
[apache-no-dotenv]
enabled = true
port = http,https
filter = apache-no-dotenv
logpath = /var/log/apache2/access.log
maxretry = 1
findtime = 3600
bantime = 86400
systemctl restart fail2ban
fail2ban-client status apache-no-dotenv
8) LXC hardening notes for this CT
You need nesting=1
for Docker. Still, keep it tight:
- Make sure
unprivileged
containers for future workloads when possible. This CT is privileged; restrict mounts and capabilities. - Proxmox firewall: allow only required inbound ports (80/443, 30000/31000 if truly public, 6333/5678 if needed). Block everything else.
- Within CT, restrict Apache to bind only on expected interfaces.
- Consider
link_down=1
during incident response to pause egress.
9) Quick verification checklist
-
curl -I / .env
returns 403/404. -
/var/www/html/flast/.env
does not exist (moved & shredded) or is600
and outside DocumentRoot. - Services load env from
/etc/flast/flast.env
and run OK. - No process exposes secrets in
/proc/*/environ
. - Apache/Nginx denies dotfiles;
Options -Indexes
set. - Backups do not include plaintext secrets.
- All leaked keys have been rotated.
___________ Refined _____________if 200 OK in Test
What your test shows
curl -sI http://127.0.0.1/.env
? 404 Not Found (HTTP vhost not serving it)curl -skI https://127.0.0.1/.env
? 200 OK (HTTPS vhost is serving it)
So yes, the .env
is currently accessible over HTTPS from the container. We’ll fix it without changing ownership to root
.
0) Inspect: which vhost & where is DocumentRoot
apachectl -S | sed -n '1,120p'
# Find the active HTTPS vhost and its DocumentRoot
grep -R "^\s*DocumentRoot\b\|^\s*Alias\b" /etc/apache2/sites-enabled/ -n
# Confirm where the file lives and its perms
stat -c '%a %U %G %n' /var/www/html/flast/.env
- If your HTTPS vhost
DocumentRoot
is/var/www/html/flast
, then/.env
maps to/var/www/html/flast/.env
.
1) Immediate mitigation (no owner change)
Keep owner as flast:flast
, but remove world-read and group-read by webserver users.
# Keep owner as-is (flast). Just tighten mode so Apache (www-data) cannot read it.
chmod 0640 /var/www/html/flast/.env
# If group is not flast, set it to flast so only user+group flast can read:
chgrp flast /var/www/html/flast/.env
# Re-test locally
curl -sI http://127.0.0.1/.env | head -n1 # expect 404 as before
curl -skI https://127.0.0.1/.env | head -n1 # expect **403 Forbidden** now (Apache can't read)
If you still get 200 OK after chmod 0640
, your app or a reverse proxy is serving it via a route. Proceed to step 2.
2) Web-server deny rule for dotfiles (keeps apps happy)
Create a small Apache conf that blocks any dotfile (including .env
) but allows /.well-known/
for ACME.
cat >/etc/apache2/conf-available/hide-dotfiles.conf <<'EOF'
# Deny requests for any dotfile (.env, .git, .ht*, etc.)
# but keep ACME HTTP-01 working (.well-known).
<Directory "/var/www/html">
<FilesMatch "^\.(?!well-known/)">
Require all denied
</FilesMatch>
</Directory>
EOF
# Adjust the Directory path above if your HTTPS vhost has a different DocumentRoot.
# Enable and reload
a2enconf hide-dotfiles
apachectl configtest && systemctl reload apache2
# Verify
curl -skI https://127.0.0.1/.env | head -n1 # expect 403 or 404
If you use per-site vhost files, you can put the<Directory>
block directly into that site’s:443
vhost instead of a global conf.
3) (Recommended) Move secrets out of web root (still no owner change)
Keeping secrets in a served tree is risky even when denied. Prefer a location outside the DocumentRoot (owner/group flast:flast
, mode 0600
or 0640
).
install -o flast -g flast -m 0600 /var/www/html/flast/.env /etc/flast/flast.env
# Leave a placeholder so nothing breaks, but empty it to avoid accidental reads
: > /var/www/html/flast/.env && chmod 000 /var/www/html/flast/.env
Wire your app to the new path in one of these ways:
A) python-dotenv inside your app entrypoint
from dotenv import load_dotenv
load_dotenv('/etc/flast/flast.env')
B) systemd unit for uvicorn/streamlit
# /etc/systemd/system/flast.server.service (example)
[Service]
User=flast
Group=flast
EnvironmentFile=/etc/flast/flast.env
# ExecStart=... (your current command)
# Then:
systemctl daemon-reload
systemctl restart flast.server
C) Docker (if the app runs in a container)
# Add to your docker run / compose:
--env-file /etc/flast/flast.env
4) Check if it was ever fetched (forensics)
# Apache access logs (both current and rotated)
awk '$7 ~ /^\/.env$/ {print}' /var/log/apache2/access.log*
# If behind reverse proxy/stunnel, also search their logs
journalctl -u stunnel --since '7 days ago' | grep -F ' /.env '
If you find hits, rotate/replace the API key that was exposed.
5) Quick verification checklist
-
chmod 0640
(or0600
) on the real secrets file, owner/groupflast:flast
. -
curl -skI https://127.0.0.1/.env
returns 403/404, not 200. - Apache conf added & reloaded (or vhost updated) with the dotfile deny.
- (Recommended) App loads secrets from
/etc/flast/flast.env
(outside webroot). - Keys rotated if any access was observed in logs.