156 lines
5.3 KiB
Markdown
156 lines
5.3 KiB
Markdown
# CLAUDE.md
|
|
|
|
This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
|
|
|
|
## Development Commands
|
|
|
|
```bash
|
|
# Install dependencies
|
|
npm run install:all
|
|
|
|
# Development (run in separate terminals)
|
|
npm run dev:backend # Starts on :3001 with tsx watch
|
|
npm run dev:frontend # Starts on :3000 with Vite HMR
|
|
|
|
# Within backend/ directory
|
|
cd backend
|
|
npm run dev # Development with tsx watch
|
|
npm run build # TypeScript compilation
|
|
npm start # Run compiled version
|
|
|
|
# Within frontend/ directory
|
|
cd frontend
|
|
npm run dev # Vite dev server
|
|
npm run build # TypeScript + Vite build
|
|
npm run preview # Preview production build
|
|
```
|
|
|
|
## Architecture Overview
|
|
|
|
VidRip is a monorepo with separate backend (Express/SQLite) and frontend (React) apps.
|
|
|
|
### Core Scheduler Logic
|
|
|
|
The scheduler in `backend/src/services/scheduler.ts` operates on two separate timers:
|
|
|
|
1. **Channel Checker**: Cron job (`0 * * * *`) runs every hour to check active channels for new videos
|
|
2. **Download Cycle**: Self-scheduling recursive setTimeout that:
|
|
- Downloads one pending video
|
|
- Calculates next run: `intervalHours * 60 ± random(varianceMinutes)`
|
|
- Schedules itself again with the calculated delay
|
|
|
|
The variance prevents predictable download patterns. The scheduler uses a single global `isDownloading` flag to prevent concurrent downloads.
|
|
|
|
### Database Layer
|
|
|
|
SQLite database (`backend/data.db`) with three tables managed via `better-sqlite3`:
|
|
- `channels`: YouTube channels to monitor
|
|
- `videos`: Individual videos with status tracking (pending/downloading/completed/failed)
|
|
- `config`: Key-value store for app settings
|
|
|
|
All database operations are in `backend/src/db/database.ts` using synchronous better-sqlite3 API. The database auto-initializes on first run.
|
|
|
|
### yt-dlp Integration
|
|
|
|
`backend/src/services/ytdlp.ts` wraps the yt-dlp CLI (must be installed on system):
|
|
- Uses `child_process.spawn` to call yt-dlp
|
|
- `--dump-json` extracts metadata without downloading
|
|
- `--flat-playlist` lists channel videos efficiently
|
|
- Progress tracking via stdout parsing (looks for percentage patterns)
|
|
- Downloads merge best video+audio to MP4 format
|
|
|
|
### Frontend Proxy Setup
|
|
|
|
Vite development server (port 3000) proxies to backend (port 3001):
|
|
- `/api/*` → Backend API endpoints
|
|
- `/downloads/*` → Static video files
|
|
|
|
This is configured in `frontend/vite.config.ts`. Production deployments need a reverse proxy (nginx, etc.).
|
|
|
|
### API Structure
|
|
|
|
Routes are split by domain:
|
|
- `backend/src/routes/channels.ts`: Channel CRUD + refresh
|
|
- `backend/src/routes/videos.ts`: Video CRUD + retry
|
|
- `backend/src/routes/config.ts`: Settings + scheduler control
|
|
|
|
Video files are served statically from `backend/downloads/` directory.
|
|
|
|
## Key Constraints
|
|
|
|
- Only 1 concurrent download supported (hardcoded in scheduler)
|
|
- Download progress is ephemeral (not persisted to DB, only in-memory)
|
|
- No authentication/authorization
|
|
- yt-dlp must be in PATH (external dependency)
|
|
- Database uses WAL mode by default for better concurrency
|
|
|
|
## Important Implementation Notes
|
|
|
|
### Scheduler Restart Behavior
|
|
|
|
When config changes (e.g., interval or variance), call `restartScheduler()` from `backend/src/services/scheduler.ts`. This stops the cron task and clears the setTimeout chain, then restarts with new settings.
|
|
|
|
### Video Status State Machine
|
|
|
|
Videos progress through states: `pending → downloading → completed/failed`
|
|
- Retry functionality resets `failed` back to `pending`
|
|
- Deleting a channel cascades to delete all its videos and files
|
|
|
|
### Channel Refresh
|
|
|
|
Adding a channel fetches ALL videos from that channel using yt-dlp's `--flat-playlist`. Large channels (1000+ videos) may take time. Videos are created in `pending` state and downloaded according to the schedule.
|
|
|
|
## File Locations
|
|
|
|
- Database: `backend/data.db`
|
|
- Downloaded videos: `backend/downloads/{videoId}.mp4`
|
|
- Environment: Optional `backend/.env` (PORT, NODE_ENV)
|
|
|
|
## System Requirements
|
|
|
|
- Node.js 18+
|
|
- yt-dlp installed and in PATH (`pip install yt-dlp` or `brew install yt-dlp`)
|
|
|
|
## Troubleshooting 403 Errors
|
|
|
|
If you encounter 403 Forbidden errors when downloading videos:
|
|
|
|
### Built-in Mitigations
|
|
|
|
The app automatically includes these anti-403 measures:
|
|
- Browser-like User-Agent headers
|
|
- YouTube referer
|
|
- Multiple retry attempts (5 extractor retries, 10 connection retries)
|
|
- Request throttling (1-5 second delays between requests)
|
|
- Fragment retries for long videos
|
|
|
|
### Additional Solution: Browser Cookies
|
|
|
|
For persistent 403 errors, export cookies from your browser:
|
|
|
|
1. **Install browser extension**:
|
|
- Chrome/Edge: "Get cookies.txt LOCALLY"
|
|
- Firefox: "cookies.txt"
|
|
|
|
2. **Export YouTube cookies**:
|
|
- Visit youtube.com while logged in
|
|
- Click extension icon and export cookies
|
|
- Save as `cookies.txt`
|
|
|
|
3. **Add to VidRip**:
|
|
- Place `cookies.txt` in `backend/` directory
|
|
- VidRip will automatically use it if present
|
|
- No restart needed
|
|
|
|
4. **Keep cookies fresh**:
|
|
- YouTube cookies expire periodically
|
|
- Re-export if 403 errors return
|
|
- Consider using an account with YouTube Premium to avoid restrictions
|
|
|
|
### Other Tips
|
|
|
|
- Update yt-dlp: `pip install --upgrade yt-dlp` (YouTube changes frequently)
|
|
- Increase download interval to avoid rate limiting
|
|
- Use variance to randomize download times
|
|
- Avoid downloading too many videos from the same channel quickly
|