Add a vault for saving whole forwarded posts
Completes TODO #5. Forwarding a post now asks whether to save it whole or extract its media; sending media directly is unchanged. Built on forwarding rather than copying file_ids. The bot already has the post in its private chat with the owner, so it stores message ids and forwards them back. forwardMessages takes a list and keeps an album intact, which is what made the caption-plus-album case tractable; ids must be strictly ascending, so they are sorted on the way out. - Vault rows live in their own table and every query is scoped to an owner_id, so a guessed id cannot reach someone else's saved post. Nothing in the vault is searchable inline, moderated or federated, and the separate table makes that structural rather than a flag. - Albums arrive as separate messages with no completion signal, so they are buffered by media_group_id behind a 1s debounce and prompt once. - The add path now takes a list of items, so extracting an album stores every piece under one set of keywords instead of asking repeatedly. - /vault searches keywords and the original caption, since the point of a vault is finding something you never got round to tagging. - Retrieval failures are reported: a message id is a pointer into the chat, not a self-contained file_id, so clearing history breaks it. Three near-identical keyword collectors are now one helper. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
@@ -7,10 +7,10 @@ Progress on branch `develop` (not pushed):
|
||||
- [x] **#6** full media coverage — `efc2be5`
|
||||
- [x] **#3** moderation queue — `0a8c81d`
|
||||
- [x] **#2** public mode + membership gate — `82b5da4`
|
||||
- [ ] **#5** save vs. extract (vault)
|
||||
- [x] **#5** save vs. extract (vault) — forward-based
|
||||
- [ ] **#4** federation
|
||||
|
||||
Test suites: `test_db.py`, `test_bot.py`, `test_access.py` — 20 cases, all passing.
|
||||
Test suites: `test_db.py`, `test_bot.py`, `test_access.py`, `test_vault.py` — 29 cases, all passing.
|
||||
|
||||
---
|
||||
|
||||
@@ -98,29 +98,45 @@ With `PUBLIC_MODE=true`, non-admin submissions land as `status='pending'` from #
|
||||
|
||||
---
|
||||
|
||||
## #5 — Save vs. extract on forward
|
||||
## #5 — Save vs. extract on forward — **implemented**
|
||||
|
||||
On a forwarded message, ask with inline buttons: **[📥 Save whole post]** / **[✂️ Extract media]**. Extract = today's behavior. Save = the private vault.
|
||||
Built on forwarding rather than the file_id copy originally planned here, which
|
||||
removed most of the work.
|
||||
|
||||
**Albums are the hard part.** A post with caption + multiple media arrives as *several separate messages* sharing a `media_group_id`, with no "album complete" signal. Standard approach: buffer by `media_group_id` and debounce ~1s after the last message before prompting. Ask once per album, not once per file.
|
||||
**The bot already holds the post.** When you forward something, Telegram puts a
|
||||
copy in your private chat with the bot, so the bot can forward *that* back later
|
||||
by `message_id`. It never needs to reach the original channel, store media, or
|
||||
re-upload anything.
|
||||
|
||||
**Schema (migration v4)** — a separate table, not a `media` column, since one vault item holds N files:
|
||||
**`forwardMessages` (plural) preserves albums.** It takes up to 100 message ids
|
||||
and returns them as one intact album, so the caption + N media case that made
|
||||
this item large is handled by Telegram itself. Ids must be strictly ascending.
|
||||
|
||||
**One table, not two:**
|
||||
|
||||
```sql
|
||||
vault_items(id, owner_id, caption TEXT, keywords TEXT, source_chat TEXT, created_at)
|
||||
vault_files(item_id, file_id, file_unique_id, media_type, position)
|
||||
vault_items(id, owner_id, chat_id, message_ids TEXT, caption, keywords, created_at)
|
||||
```
|
||||
|
||||
Vault rows are **always private** and never touched by the federation sync in #4 — enforce that structurally by keeping them in a different table, so no `visibility` flag can ever be mis-set.
|
||||
Every vault query takes an `owner_id` and filters on it, so a guessed id cannot
|
||||
reach another user's post. Vault rows live in their own table and are never
|
||||
searched by inline mode, never moderated, and never federated.
|
||||
|
||||
**Search should cover the caption, not just keywords** — the point of a vault is finding a post you saved without having tagged it. This is where the current `LIKE '%,kw%'` approach (`db.py:43`) stops being adequate; use an FTS5 virtual table over `caption + keywords`.
|
||||
**Flow.** A forwarded message prompts *Save whole post* / *Extract media*
|
||||
(extract only appears when there is media). Save collects keywords and stores
|
||||
the pointer; extract runs the existing add path, now generalised to a list so a
|
||||
whole album shares one set of keywords. Albums are buffered by `media_group_id`
|
||||
with a 1s debounce, since Telegram gives no "album complete" signal.
|
||||
|
||||
**Open question — recall UX.** Telegram **inline mode cannot send an album**; one inline result sends exactly one message. So a multi-file vault item can't come back through `@bot query` intact. Two options:
|
||||
**Recall is inside the bot only**, as decided: `/vault [search]` lists matching
|
||||
posts with a retrieve button and a delete button. Matching covers keywords and
|
||||
the original caption. The save confirmation tells the user plainly that vault
|
||||
posts do not appear in inline search.
|
||||
|
||||
1. Inline returns the item's *first* media with the caption attached — works everywhere, but loses the other files.
|
||||
2. A `/vault <query>` command in the bot's private chat that replies with the full album, which you then forward manually — preserves everything, costs one extra step.
|
||||
|
||||
These aren't exclusive; doing both (1 for speed, 2 for fidelity) is probably right, but it's your call. Single-file vault items are unaffected either way.
|
||||
**The tradeoff accepted:** a `message_id` is a pointer into the bot's chat with
|
||||
the owner, not self-contained like a `file_id`. If that history is cleared, the
|
||||
pointer breaks; retrieval catches the failure and tells the user to forward the
|
||||
post again.
|
||||
|
||||
---
|
||||
|
||||
@@ -164,14 +180,14 @@ The hub is a separate deployable (small FastAPI + SQLite would do). Do you want
|
||||
| 6 | Full media coverage | 0 (only for the refactor) | S |
|
||||
| 3 | Moderation queue | 0 | M |
|
||||
| 2 | Public mode + membership gate | 0, 3 | M |
|
||||
| 5 | Save vs. extract + vault | 0 | L |
|
||||
| 5 | Save vs. extract + vault | 0 | M (was L) |
|
||||
| 4 | Federation | 0, 3, hub built | XL |
|
||||
|
||||
#6 is shippable on its own and touches nothing else — good first commit on `develop`. #4 should stay last: it's the only item that's hard to reverse once other people are running instances against your hub.
|
||||
|
||||
## Decisions — settled 2026-09-01
|
||||
|
||||
1. **#5 recall UX** — inline cannot send an album. For multi-file vault items, the inline result sends the **link to the original post** where one exists; otherwise it tells the user to open the bot and pull the full album there. Single-file items send directly as today.
|
||||
1. **#5 recall UX** — settled further during implementation: the vault is **forward-based and retrievable only inside the bot**. Inline search does not surface vault posts at all; the save confirmation says so.
|
||||
2. **#4 hub location** — **separate repository.** Channel name and hub hosting are the owner's call, not decided here.
|
||||
3. **#6 video_note** — **store as `video`**, accepting the loss of round presentation, so round videos stay inline-searchable. See the risk note below.
|
||||
4. **#4 architecture** — **shared federation channel accepted.** Media rides the channel, metadata rides the hub API.
|
||||
|
||||
Reference in New Issue
Block a user