Initial
Takes in a file, process it and output a new file with additional columns.
This commit is contained in:
@@ -1,2 +1,70 @@
|
||||
# Warner-Vistex-Automater
|
||||
|
||||
Stage 1 of a Bootstrap-based PHP app that uploads an Excel workbook, processes the rows, stores the imported data in PostgreSQL, and previews the imported rows in a paginated table.
|
||||
|
||||
## Features
|
||||
|
||||
- Upload `.xls` or `.xlsx` files.
|
||||
- Validate required spreadsheet headers before import.
|
||||
- Wipe previous imported rows before each new import.
|
||||
- Compute the stage 1 derived fields during import.
|
||||
- Store normalized imported data plus calculated data in PostgreSQL using `jsonb`.
|
||||
- Process large imports in short chunks to avoid request timeouts.
|
||||
- Review imported rows in a Bootstrap table with pagination.
|
||||
- Show upload progress and import status in the browser.
|
||||
|
||||
## Requirements
|
||||
|
||||
- PHP 8.3 or newer
|
||||
- Composer
|
||||
- PostgreSQL
|
||||
|
||||
## Setup
|
||||
|
||||
1. Edit `config.php` and update the database settings if your host names differ from the defaults.
|
||||
- Or set `DATABASE_URL`, for example:
|
||||
|
||||
```bash
|
||||
DATABASE_URL=postgres://postgres:postgres@host.docker.internal:5432/devdb?sslmode=prefer
|
||||
```
|
||||
|
||||
- The default host is `localhost` for CLI scripts and `postgres_db` for the web app.
|
||||
- If your setup differs, `DATABASE_URL` is the easiest override.
|
||||
- Set `APP_URL` if the app will run from a different folder or base URL than `http://localhost/warner/`.
|
||||
2. Install dependencies:
|
||||
|
||||
```bash
|
||||
composer install
|
||||
```
|
||||
|
||||
3. Run the database migration:
|
||||
|
||||
```bash
|
||||
composer run migrate
|
||||
```
|
||||
|
||||
4. Start the app:
|
||||
|
||||
```bash
|
||||
php -S 127.0.0.1:8000 -t public
|
||||
```
|
||||
|
||||
5. Open [http://127.0.0.1:8000](http://127.0.0.1:8000).
|
||||
|
||||
## Import flow
|
||||
|
||||
1. The app validates that a file was uploaded.
|
||||
2. The file must be an Excel workbook.
|
||||
3. Required columns are checked by header name.
|
||||
4. Existing imported rows are deleted in a transaction.
|
||||
5. The workbook rows are normalized and derived fields are calculated.
|
||||
6. The new rows are inserted and then displayed in the preview table.
|
||||
|
||||
## Data model
|
||||
|
||||
The schema uses two tables:
|
||||
|
||||
- `import_batches` stores metadata for each import run.
|
||||
- `import_rows` stores the row payloads using `jsonb` for original, normalized, and calculated data.
|
||||
|
||||
This keeps the model flexible for later import stages and additional derived fields.
|
||||
|
||||
Reference in New Issue
Block a user