Fix ClamAV account setup and add install verification

README.md

@@ -1,22 +1,82 @@
 # malware-watch installer
 
-Files:
+`malware-watch` combines process/network behavior detection, abuse.ch ThreatFox/MalwareBazaar IOC feeds, confirmed-threat automatic quarantine, persistent IOC firewall blocking, and optional ClamAV scanning.
-- `install.sh` — installs malware-watch, abuse.ch feeds, automatic quarantine, IOC firewall blocking, and optional official ClamAV.
+
-- `.env` — private Discord Webhook / abuse.ch Auth-Key and tuning values.
+## Files
+
+- `install.sh` — installs malware-watch, abuse.ch feed updater, automatic quarantine, IOC firewall restore, systemd timers, and optional official ClamAV.
+- `verify.sh` — verifies that the installation is healthy.
+- `.env` — installer configuration. **Replace the placeholder values locally before installation.**
+
+The repository version of `.env` contains placeholders only. Never commit a real Discord webhook or abuse.ch Auth-Key.
+
+## Install
 
-Install:
 ```bash
-cd /app/malware-watch
+git clone https://github.com/javara999/malware-watch.git
+cd malware-watch
+chmod +x install.sh verify.sh
+nano .env
 sudo ./install.sh
 ```
 
-Useful logs:
+Required `.env` values:
+
 ```bash
-journalctl -u malware-watch.service -f
+DISCORD_WEBHOOK='디스코드웹훅URL'
-journalctl -u malware-feed-update.service
+ABUSECH_AUTH_KEY='abuse.ch api 키'
-journalctl -u clamav-risk-scan.service
-journalctl -u clamav-deep-scan.service
-journalctl -u clamav-freshclam-local.service
 ```
 
-The `.env` file is mode 600 and is ignored by Git.
+Replace both placeholder strings with your real values only on the server where you are installing.
+
+The installer is safe to rerun after a partial/failed installation. Existing malware-watch files are backed up under `/root/security-evidence/malware-watch/install-backup-*` before replacement.
+
+## Verify installation
+
+```bash
+sudo ./verify.sh
+```
+
+It checks required files/permissions, malware-watch and abuse.ch timers, ThreatFox/MalwareBazaar feeds, ClamAV installation/account/signature DB/timers, and a real one-shot execution of `malware-watch.py`.
+
+A healthy installation ends with:
+
+```text
+Summary: PASS=<number> WARN=0 FAIL=0
+```
+
+A `WARN` for the ClamAV signature DB immediately after installation can be normal: `freshclam` may still be waiting for its first successful download. `FAIL` means something should be fixed.
+
+To also send a harmless Discord verification message:
+
+```bash
+sudo ./verify.sh --test-discord
+```
+
+## Manual status checks
+
+```bash
+systemctl status malware-watch.timer --no-pager
+systemctl status malware-feed-update.timer --no-pager
+systemctl status clamav-freshclam-local.timer --no-pager
+systemctl status clamav-risk-scan.timer --no-pager
+systemctl status clamav-deep-scan.timer --no-pager
+systemctl list-timers --all --no-pager | grep -E 'malware-watch|malware-feed-update|clamav-'
+```
+
+Useful logs:
+
+```bash
+journalctl -u malware-watch.service -n 100 --no-pager
+journalctl -u malware-feed-update.service -n 100 --no-pager
+journalctl -u clamav-risk-scan.service -n 100 --no-pager
+journalctl -u clamav-deep-scan.service -n 100 --no-pager
+journalctl -u clamav-freshclam-local.service -n 100 --no-pager
+```
+
+## Security notes
+
+- Confirmed threats are quarantined; ordinary heuristic/high-score findings are not automatically deleted.
+- Quarantine/evidence is stored under `/root/security-evidence/`.
+- `.env` should be mode `600` on an installed server.
+- Keep real webhook/API credentials out of Git history.

install.sh

@@ -39,7 +39,7 @@ command -v apt-get >/dev/null || die "This installer currently supports Debian/U
 log "Installing base dependencies"
 apt-get update -y
 DEBIAN_FRONTEND=noninteractive apt-get install -y \
-  ca-certificates curl python3 iproute2 procps util-linux iptables coreutils
+  ca-certificates curl python3 iproute2 procps util-linux iptables coreutils passwd
 
 install -d -m 700 /etc/malware-watch /var/lib/malware-watch /var/lib/malware-watch/feeds
 install -d -m 700 /root/security-evidence/malware-watch /root/security-evidence/quarantine
@@ -481,6 +481,15 @@ if [[ "$INSTALL_CLAMAV" == "1" ]]; then
 
   [[ -x /usr/local/bin/clamscan ]] || die "Official ClamAV clamscan not found after installation"
   [[ -f /usr/local/etc/freshclam.conf.sample ]] || die "freshclam.conf.sample not found"
+
+  # Official ClamAV packages can be present without a pre-created service account.
+  # Ensure the account exists before assigning ownership to the signature DB directory.
+  if ! getent group clamav >/dev/null 2>&1; then
+    groupadd --system clamav
+  fi
+  if ! id -u clamav >/dev/null 2>&1; then
+    useradd --system --gid clamav --home-dir /var/lib/clamav --shell /usr/sbin/nologin clamav
+  fi
   install -d -o clamav -g clamav -m 755 /var/lib/clamav
   cp /usr/local/etc/freshclam.conf.sample /usr/local/etc/freshclam.conf
   sed -i 's/^Example/#Example/' /usr/local/etc/freshclam.conf

verify.sh

@@ -0,0 +1,136 @@
+#!/usr/bin/env bash
+set -u
+
+PASS=0
+WARN=0
+FAIL=0
+
+ok(){ printf "[PASS] %s\n" "$*"; PASS=$((PASS+1)); }
+warn(){ printf "[WARN] %s\n" "$*"; WARN=$((WARN+1)); }
+fail(){ printf "[FAIL] %s\n" "$*"; FAIL=$((FAIL+1)); }
+
+if [[ ${EUID:-$(id -u)} -ne 0 ]]; then
+  exec sudo bash "$0" "$@"
+fi
+
+TEST_DISCORD=0
+[[ "${1:-}" == "--test-discord" ]] && TEST_DISCORD=1
+
+printf "malware-watch verification | host=%s | time=%s\n\n" "$(hostname)" "$(date -Is)"
+
+for f in \
+  /etc/malware-watch/config \
+  /usr/local/sbin/malware-watch.py \
+  /usr/local/sbin/malware-quarantine.py \
+  /usr/local/sbin/malware-feed-update.py \
+  /usr/local/sbin/clamav-watch-scan.sh \
+  /usr/local/sbin/block-malware-iocs; do
+  if [[ -f "$f" ]]; then ok "file exists: $f"; else fail "missing file: $f"; fi
+done
+
+if [[ -f /etc/malware-watch/config ]]; then
+  mode=$(stat -c %a /etc/malware-watch/config 2>/dev/null || true)
+  [[ "$mode" == "600" ]] && ok "config permissions are 600" || warn "config permissions are $mode (expected 600)"
+  if grep -q '^DISCORD_WEBHOOK=' /etc/malware-watch/config && grep -q '^ABUSECH_AUTH_KEY=' /etc/malware-watch/config; then
+    ok "Discord webhook and abuse.ch key entries exist"
+  else
+    fail "Discord webhook or abuse.ch key entry is missing"
+  fi
+fi
+
+for t in malware-watch.timer malware-feed-update.timer; do
+  if systemctl is-enabled "$t" >/dev/null 2>&1 && systemctl is-active "$t" >/dev/null 2>&1; then
+    ok "$t is enabled and active"
+  else
+    fail "$t is not enabled/active"
+  fi
+done
+
+if systemctl is-enabled block-malware-iocs.service >/dev/null 2>&1 && systemctl is-active block-malware-iocs.service >/dev/null 2>&1; then
+  ok "block-malware-iocs.service is enabled and active"
+else
+  fail "block-malware-iocs.service is not enabled/active"
+fi
+
+TF=/var/lib/malware-watch/feeds/threatfox.json
+MB=/var/lib/malware-watch/feeds/malwarebazaar.json
+if [[ -s "$TF" ]]; then
+  n=$(python3 - "$TF" <<'PY'
+import json,sys
+try:
+ d=json.load(open(sys.argv[1])); print(len(d.get("iocs",{})))
+except Exception: print(-1)
+PY
+)
+  [[ "$n" =~ ^[0-9]+$ ]] && (( n > 0 )) && ok "ThreatFox feed loaded: $n IOCs" || warn "ThreatFox feed exists but contains no parsed IOCs"
+else
+  fail "ThreatFox feed missing/empty"
+fi
+
+if [[ -s "$MB" ]]; then
+  n=$(python3 - "$MB" <<'PY'
+import json,sys
+try:
+ d=json.load(open(sys.argv[1])); print(len(d.get("hashes",{})))
+except Exception: print(-1)
+PY
+)
+  [[ "$n" =~ ^[0-9]+$ ]] && (( n > 0 )) && ok "MalwareBazaar feed loaded: $n hashes" || warn "MalwareBazaar feed exists but contains no parsed hashes"
+else
+  fail "MalwareBazaar feed missing/empty"
+fi
+
+if [[ -x /usr/local/bin/clamscan ]]; then
+  ok "ClamAV installed: $(/usr/local/bin/clamscan --version 2>/dev/null | head -n1)"
+  if id -u clamav >/dev/null 2>&1; then ok "clamav service account exists"; else fail "clamav service account is missing"; fi
+  if compgen -G '/var/lib/clamav/*.cvd' >/dev/null || compgen -G '/var/lib/clamav/*.cld' >/dev/null; then
+    ok "ClamAV signature database exists"
+  else
+    warn "ClamAV signature DB is not ready yet; freshclam timer may still need to download it"
+  fi
+  for t in clamav-freshclam-local.timer clamav-risk-scan.timer clamav-deep-scan.timer; do
+    if systemctl is-enabled "$t" >/dev/null 2>&1 && systemctl is-active "$t" >/dev/null 2>&1; then
+      ok "$t is enabled and active"
+    else
+      fail "$t is not enabled/active"
+    fi
+  done
+else
+  warn "ClamAV is not installed (INSTALL_CLAMAV may have been disabled)"
+fi
+
+if [[ -x /usr/local/sbin/malware-watch.py ]]; then
+  out=$(/usr/local/sbin/malware-watch.py 2>&1); rc=$?
+  if (( rc == 0 )); then
+    ok "malware-watch one-shot execution succeeded: $out"
+  else
+    fail "malware-watch one-shot execution failed (rc=$rc): $out"
+  fi
+fi
+
+if (( TEST_DISCORD )); then
+  if [[ -r /etc/malware-watch/config ]]; then
+    set -a
+    . /etc/malware-watch/config
+    set +a
+    payload=$(python3 - <<'PY'
+import json,socket
+print(json.dumps({"content":f"malware-watch verify OK | server: {socket.gethostname()}"},ensure_ascii=False))
+PY
+)
+    code=$(curl -sS -o /tmp/malware-watch-discord-verify.out -w '%{http_code}' \
+      -A 'Mozilla/5.0 (compatible; malware-watch/verify)' \
+      -H 'Content-Type: application/json' --data "$payload" "$DISCORD_WEBHOOK" || true)
+    if [[ "$code" == "204" || "$code" == "200" ]]; then
+      ok "Discord webhook test succeeded (HTTP $code)"
+    else
+      fail "Discord webhook test failed (HTTP ${code:-000})"
+    fi
+    rm -f /tmp/malware-watch-discord-verify.out
+  else
+    fail "cannot test Discord: config unavailable"
+  fi
+fi
+
+printf "\nSummary: PASS=%d WARN=%d FAIL=%d\n" "$PASS" "$WARN" "$FAIL"
+(( FAIL == 0 ))