Takes in a file, process it and output a new file with additional columns.
This commit is contained in:
Stefan Lazic
2026-07-10 15:45:02 +01:00
parent 2f387a4c1c
commit a451060c1b
876 changed files with 169373 additions and 0 deletions
+68
View File
@@ -1,2 +1,70 @@
# Warner-Vistex-Automater
Stage 1 of a Bootstrap-based PHP app that uploads an Excel workbook, processes the rows, stores the imported data in PostgreSQL, and previews the imported rows in a paginated table.
## Features
- Upload `.xls` or `.xlsx` files.
- Validate required spreadsheet headers before import.
- Wipe previous imported rows before each new import.
- Compute the stage 1 derived fields during import.
- Store normalized imported data plus calculated data in PostgreSQL using `jsonb`.
- Process large imports in short chunks to avoid request timeouts.
- Review imported rows in a Bootstrap table with pagination.
- Show upload progress and import status in the browser.
## Requirements
- PHP 8.3 or newer
- Composer
- PostgreSQL
## Setup
1. Edit `config.php` and update the database settings if your host names differ from the defaults.
- Or set `DATABASE_URL`, for example:
```bash
DATABASE_URL=postgres://postgres:postgres@host.docker.internal:5432/devdb?sslmode=prefer
```
- The default host is `localhost` for CLI scripts and `postgres_db` for the web app.
- If your setup differs, `DATABASE_URL` is the easiest override.
- Set `APP_URL` if the app will run from a different folder or base URL than `http://localhost/warner/`.
2. Install dependencies:
```bash
composer install
```
3. Run the database migration:
```bash
composer run migrate
```
4. Start the app:
```bash
php -S 127.0.0.1:8000 -t public
```
5. Open [http://127.0.0.1:8000](http://127.0.0.1:8000).
## Import flow
1. The app validates that a file was uploaded.
2. The file must be an Excel workbook.
3. Required columns are checked by header name.
4. Existing imported rows are deleted in a transaction.
5. The workbook rows are normalized and derived fields are calculated.
6. The new rows are inserted and then displayed in the preview table.
## Data model
The schema uses two tables:
- `import_batches` stores metadata for each import run.
- `import_rows` stores the row payloads using `jsonb` for original, normalized, and calculated data.
This keeps the model flexible for later import stages and additional derived fields.