From 0732f2c34b9ccd6b0a28c99a679af58765149342 Mon Sep 17 00:00:00 2001 From: goyban Date: Tue, 1 Sep 2026 22:09:37 +0000 Subject: [PATCH] Record what may be shared, ahead of federation The bot-side half of TODO #4. Nothing federates yet; this establishes what would be eligible to, and proves the boundary before there is a hub that could leak across it. - Schema v5 adds visibility, defaulting to private. Upgrading an existing library therefore opts nothing into sharing, which a test on the legacy upgrade path pins down. - list_syncable() is the only function that will ever feed a hub, and it requires both public and approved. Tests force rows into states set_visibility refuses, so the filter is what protects the boundary rather than the caller being careful. - set_visibility refuses anything not approved: media still in the moderation queue cannot be marked shared. - /share lists memes with their current state and asks before making one public, since un-sharing stops future copies but cannot recall those already taken. The wording describes what sharing will mean once a hub exists, because by then warning is too late. - /list shows the same markers. Vault posts have no path into the sync set at all; being in a separate table is what guarantees it, and there is a test saying so. Co-Authored-By: Claude Opus 5 --- PLAN.md | 16 +++++++- README.fa.md | 11 ++++++ README.md | 11 ++++++ VERSION | 2 +- bot.py | 103 ++++++++++++++++++++++++++++++++++++++++++++++++++- db.py | 55 ++++++++++++++++++++++++++- test_db.py | 81 +++++++++++++++++++++++++++++++++++++++- 7 files changed, 272 insertions(+), 7 deletions(-) diff --git a/PLAN.md b/PLAN.md index 57b38a2..ca7065d 100644 --- a/PLAN.md +++ b/PLAN.md @@ -8,9 +8,9 @@ Progress on branch `develop` (not pushed): - [x] **#3** moderation queue — `0a8c81d` - [x] **#2** public mode + membership gate — `82b5da4` - [x] **#5** save vs. extract (vault) — forward-based -- [ ] **#4** federation +- [~] **#4** federation — bot-side visibility done; hub, channel and sync client outstanding -Test suites: `test_db.py`, `test_bot.py`, `test_access.py`, `test_vault.py` — 29 cases, all passing. +Test suites: `test_db.py`, `test_bot.py`, `test_access.py`, `test_vault.py` — 34 cases, all passing. --- @@ -166,6 +166,18 @@ Split media transport from metadata sync: - Rate-limit per instance. A compromised peer token should be able to spam, at worst, its own rows. - Practical note: a federated pool of user-submitted media needs a takedown path for illegal content, and as the hub operator that lands on you. Worth deciding the policy before launching, not after. +### Status + +Done in this repo: the `visibility` column (private by default), `/share` to +change it, and `list_syncable()` — the single function that defines what is +ever allowed to leave this instance. It is guarded by tests that force rows +into states the writer would refuse, so the filter is proven rather than +assumed. Nothing calls it yet. + +Outstanding, and all of it depends on decisions only the owner can make: +the federation channel, the hub deployment, the instance tokens, and the sync +client that pushes to and pulls from the hub. + ### Open question The hub is a separate deployable (small FastAPI + SQLite would do). Do you want it in this repo as `hub/`, or as its own repository? It has a different release cadence and a different threat model, which argues for separate. diff --git a/README.fa.md b/README.fa.md index 8869a9b..a75024d 100644 --- a/README.fa.md +++ b/README.fa.md @@ -61,6 +61,7 @@ docker compose up -d --build | `/list` | نمایش همه رسانه‌های ذخیره‌شده | | `/vault [search]` | بازیابی پستی که کامل ذخیره کرده‌اید | | `/review` | بررسی یک‌به‌یک ارسال‌های در انتظار تأیید (فقط ادمین) | +| `/share [search]` | مشخص‌کردن اینکه یک میم به اشتراک گذاشته شود یا خصوصی بماند (فقط ادمین) | | `/delete ` | حذف یک مورد (ID را از `/list` بگیرید) | | `/cancel` | لغو عملیات جاری | @@ -110,6 +111,16 @@ REQUIRE_MODE=all > **تا وقتی مطمئن نیستید این گزینه را خاموش بگذارید.** وقتی `ALLOWED_USERS` خالی باشد همه کاربرِ مجاز حساب می‌شوند، پس روشن‌کردن این گزینه یعنی پذیرفتن رسانه از هر کسی که ربات را پیدا کند. +### اشتراک‌گذاری (مقدمات) + +هر میم به‌صورت پیش‌فرض **خصوصی** است. با `/share [search]` فهرست میم‌ها را با نشان 🔒 یا 🌐 می‌بینید و می‌توانید وضعیت هرکدام را عوض کنید. در `/list` هم همین نشان‌ها دیده می‌شود. + +> **هنوز هیچ‌چیزی به اشتراک گذاشته نمی‌شود.** این فقط خواستهٔ شما را ثبت می‌کند. فدراسیون — جایی که ربات‌هایی که جدا اجرا می‌شوند میم‌های عمومی‌شان را روی هم می‌گذارند — هنوز ساخته نشده و تا آن‌موقع هیچ میمی از این ربات بیرون نمی‌رود. متن تأیید، معنای اشتراک‌گذاری را در آینده توضیح می‌دهد، چون بعد از راه‌افتادن هاب دیگر برای هشدار دادن دیر است. + +دو قاعده در همان تک‌کوئری‌ای که قرار است به هاب داده شود اعمال می‌شود: فقط رسانه‌ای که هم **تأییدشده** و هم **عمومی** باشد واجد شرایط است. رسانه‌ای که هنوز در انتظار بررسی است اصلاً قابل اشتراک‌گذاری نیست، و پست‌های گاوصندوق در جدولی جدا هستند و هیچ مسیری به آنجا ندارند. + +خصوصی‌کردن دوباره، جلوی کپی‌های بعدی را می‌گیرد اما کپی‌هایی که قبلاً گرفته شده را برنمی‌گرداند؛ به همین دلیل قبل از عمومی‌کردن تأیید گرفته می‌شود، نه بعد از آن. + ### انتقال به سرور دیگر فقط کافی است این دو فایل را منتقل کنید: diff --git a/README.md b/README.md index 9b76782..2be6292 100644 --- a/README.md +++ b/README.md @@ -68,6 +68,7 @@ Telegram suggests these commands as you type `/`, with a short description besid | `/vault [search]` | Retrieve a whole post you saved | | `/review` | Review pending submissions one by one (admin only) | | `/edit [search]` | Select a meme and edit its keywords (admin only) | +| `/share [search]` | Mark a meme as shared or private (admin only) | | `/delete ` | Remove an entry (get ID from `/list`; admin only) | | `/cancel` | Cancel current add operation | @@ -117,6 +118,16 @@ The queue keeps no cursor. `/review` always fetches the oldest pending item, so > **Leave this off unless you mean it.** With `ALLOWED_USERS` empty, everyone is an allowed user, so turning submissions on would accept media from anyone who finds the bot. +### Sharing (groundwork) + +Every meme is **private** by default. `/share [search]` lists your memes with 🔒 or 🌐 and lets you flip one. `/list` shows the same markers. + +> **Nothing is shared yet.** This only records your intent. Federation — where independently run bots pool their shared memes — is not built, and no meme leaves this bot until it is. The wording in the confirmation describes what sharing *will* mean, because once a hub exists it is too late to start warning. + +Two rules are enforced in the one query that will ever feed a hub: only **approved** and **public** media is eligible. Media awaiting moderation cannot be marked shared at all, and vault posts are in a separate table with no path there whatsoever. + +Un-sharing stops future copies but cannot recall copies already taken, which is why the confirmation asks before making something public rather than after. + ### Moving to another server The only files you need to transfer are: diff --git a/VERSION b/VERSION index 88c5fb8..bc80560 100644 --- a/VERSION +++ b/VERSION @@ -1 +1 @@ -1.4.0 +1.5.0 diff --git a/bot.py b/bot.py index 47c2080..5d33e85 100644 --- a/bot.py +++ b/bot.py @@ -86,6 +86,7 @@ ADMIN_COMMANDS = [ *USER_COMMANDS, BotCommand(command="review", description="Review pending submissions"), BotCommand(command="edit", description="Edit meme keywords"), + BotCommand(command="share", description="Choose what is shared"), BotCommand(command="delete", description="Remove a meme by ID"), ] @@ -252,6 +253,7 @@ def menu_text(user_id: int) -> str: "/vault [search] — retrieve a whole post you saved\n" "/review — review pending submissions\n" "/edit [search] — edit keywords of a meme\n" + "/share [search] — mark a meme shared or private\n" "/delete <id> — remove a meme by ID\n" "/cancel — cancel current operation" ) @@ -447,7 +449,10 @@ async def list_page(page: int) -> tuple[str, InlineKeyboardMarkup | None]: total_pages = (total + LIST_PAGE_SIZE - 1) // LIST_PAGE_SIZE page = max(0, min(page, total_pages - 1)) items = await db.list_media(page, LIST_PAGE_SIZE) - lines = [f"{item['id']} [{item['media_type']}] — {item['keywords']}" for item in items] + lines = [ + f"{visibility_icon(item)} {item['id']} [{item['media_type']}] — {item['keywords']}" + for item in items + ] text = f"Page {page + 1} of {total_pages}\n" + "\n".join(lines) buttons = [] @@ -832,6 +837,102 @@ async def on_vault_action(callback: CallbackQuery): ) +# ── /share ─────────────────────────────────────────────────────────────────── +# +# Marking media public only records intent: nothing leaves this bot until a +# federation hub is configured. The confirmation is worded for what sharing +# will mean once one is, because by then it is too late to warn. + +def visibility_icon(item: dict) -> str: + return "🌐" if item.get("visibility") == "public" else "🔒" + + +@dp.message(Command("share")) +async def cmd_share(msg: Message): + if not is_admin(msg.from_user.id): + await msg.answer("Only admins can change what is shared.") + return + + parts = msg.text.split(maxsplit=1) + query = parts[1].strip() if len(parts) > 1 else "" + items = await db.search_media(query) + if not items: + no_match = f" matching '{query}'" if query else "" + await msg.answer(f"No memes found{no_match}.") + return + + rows = [ + [InlineKeyboardButton( + text=f"{visibility_icon(item)} #{item['id']} {item['keywords'][:35]}", + callback_data=f"share:pick:{item['id']}", + )] + for item in items[:10] + ] + public_count = await db.count_public() + await msg.answer( + f"🔒 private · 🌐 shared — tap to change.\n" + f"{public_count} of your memes are marked shared.", + reply_markup=InlineKeyboardMarkup(inline_keyboard=rows), + ) + + +@dp.callback_query(F.data.startswith("share:")) +async def on_share_action(callback: CallbackQuery): + if not is_admin(callback.from_user.id): + await callback.answer("Not allowed.", show_alert=True) + return + + _, action, raw_id = callback.data.split(":", 2) + media_id = int(raw_id) + item = await db.get_media_by_id(media_id) + if item is None: + await callback.answer("Not found.", show_alert=True) + return + + if action == "pick": + await callback.message.edit_reply_markup(reply_markup=None) + if item["visibility"] == "public": + await db.set_visibility(media_id, "private") + await callback.message.answer( + f"🔒 #{media_id} is private again. Any instance that already " + "took a copy keeps it — this stops future sharing, it does not recall " + "what has gone.", + parse_mode="HTML", + ) + await callback.answer() + return + + await callback.message.answer( + f"Share #{media_id} [{item['media_type']}]\n" + f"Keywords: {item['keywords'].replace(',', ', ')}\n\n" + "Shared media is copied by every other instance that syncs with it. " + "You can stop sharing later, but you cannot take back copies already " + "made. Nothing is shared until a federation hub is configured.", + parse_mode="HTML", + reply_markup=InlineKeyboardMarkup(inline_keyboard=[[ + InlineKeyboardButton(text="🌐 Share it", callback_data=f"share:yes:{media_id}"), + InlineKeyboardButton(text="Cancel", callback_data=f"share:no:{media_id}"), + ]]), + ) + await callback.answer() + return + + await callback.message.edit_reply_markup(reply_markup=None) + + if action == "no": + await callback.message.answer("Left private.") + await callback.answer() + return + + shared = await db.set_visibility(media_id, "public") + if shared: + await callback.message.answer(f"🌐 #{media_id} is marked shared.", parse_mode="HTML") + else: + # set_visibility refuses anything not approved. + await callback.message.answer("Only approved media can be shared.") + await callback.answer() + + # ── /review ────────────────────────────────────────────────────────────────── # Sending a stored item back needs one call per media type. Stickers cannot diff --git a/db.py b/db.py index 1f985cd..7b1da28 100644 --- a/db.py +++ b/db.py @@ -4,7 +4,7 @@ import os DB_PATH = os.getenv("DB_PATH", "/data/media.db") # Bump this and add a matching _migrate_to_vN() when the schema changes. -SCHEMA_VERSION = 4 +SCHEMA_VERSION = 5 # ── schema / migrations ────────────────────────────────────────────────────── @@ -87,10 +87,18 @@ async def _migrate_to_v4(db) -> None: await db.execute("CREATE INDEX IF NOT EXISTS idx_vault_owner ON vault_items(owner_id)") +async def _migrate_to_v5(db) -> None: + """Mark what may leave this instance. Everything defaults to private, so an + upgrade never opts an existing library into sharing.""" + await db.execute("ALTER TABLE media ADD COLUMN visibility TEXT NOT NULL DEFAULT 'private'") + await db.execute("CREATE INDEX IF NOT EXISTS idx_visibility ON media(visibility)") + + MIGRATIONS = { 2: _migrate_to_v2, 3: _migrate_to_v3, 4: _migrate_to_v4, + 5: _migrate_to_v5, } @@ -160,7 +168,7 @@ async def delete_media(media_id: int) -> bool: # ── reads ──────────────────────────────────────────────────────────────────── -COLUMNS = "id, owner_id, file_id, file_unique_id, media_type, keywords, status" +COLUMNS = "id, owner_id, file_id, file_unique_id, media_type, keywords, status, visibility" async def search_media(query: str, owner_id: int | None = None, @@ -267,6 +275,49 @@ async def set_status(media_id: int, status: str, reviewer_id: int) -> bool: return cur.rowcount > 0 +# ── visibility / federation ────────────────────────────────────────────────── + +async def set_visibility(media_id: int, visibility: str) -> bool: + """Only approved media can be shared; something still awaiting moderation + or already rejected must never become publishable.""" + async with aiosqlite.connect(DB_PATH) as db: + cur = await db.execute( + "UPDATE media SET visibility = ? WHERE id = ? AND status = 'approved'", + (visibility, media_id), + ) + await db.commit() + return cur.rowcount > 0 + + +async def list_syncable(limit: int = 500) -> list[dict]: + """The only rows permitted to leave this instance. + + Both conditions matter: private media is nobody else's business, and + unreviewed or rejected media must not be pushed to peers. Everything that + ever talks to the hub goes through this function. + """ + async with aiosqlite.connect(DB_PATH) as db: + db.row_factory = aiosqlite.Row + cursor = await db.execute( + f""" + SELECT {COLUMNS} FROM media + WHERE visibility = 'public' AND status = 'approved' + ORDER BY added_at ASC, id ASC + LIMIT ? + """, + (limit,), + ) + return [dict(r) for r in await cursor.fetchall()] + + +async def count_public() -> int: + async with aiosqlite.connect(DB_PATH) as db: + cursor = await db.execute( + "SELECT COUNT(*) FROM media WHERE visibility = 'public' AND status = 'approved'" + ) + return (await cursor.fetchone())[0] + + # ── vault ──────────────────────────────────────────────────────────────────── # # Every query here takes an owner_id and filters on it. A vault item belongs to diff --git a/test_db.py b/test_db.py index b25e4df..03a0565 100644 --- a/test_db.py +++ b/test_db.py @@ -56,6 +56,10 @@ async def test_legacy_upgrade(tmp: str): tables = {r[0] for r in c.execute("SELECT name FROM sqlite_master WHERE type='table'")} assert statuses == {"approved"}, f"upgraded rows must stay live: {statuses}" assert "vault_items" in tables, "upgrading should create the vault table too" + with sqlite3.connect(path) as c: + visibilities = {r[0] for r in c.execute("SELECT visibility FROM media")} + assert visibilities == {"private"}, \ + f"upgrading must not opt an existing library into sharing: {visibilities}" print(" legacy v1 database upgrades with ids, keywords and visibility intact") @@ -168,6 +172,78 @@ async def test_rejected_stays_hidden(tmp: str): print(" rejected items leave the queue and never surface in search") +async def test_new_media_is_private(tmp: str): + path = os.path.join(tmp, "visibility.db") + db = load_db(path) + await db.init_db() + + await db.add_media(111, "F1", "U1", "photo", ["kw"]) + assert (await db.search_media("kw"))[0]["visibility"] == "private" + assert await db.list_syncable() == [], "nothing should be shared by default" + print(" media is private unless explicitly shared") + + +async def test_only_public_and_approved_can_sync(tmp: str): + path = os.path.join(tmp, "syncable.db") + db = load_db(path) + await db.init_db() + + await db.add_media(111, "F1", "U1", "photo", ["private_approved"]) + await db.add_media(111, "F2", "U2", "photo", ["public_approved"]) + await db.add_media(111, "F3", "U3", "photo", ["public_pending"], status="pending") + await db.add_media(111, "F4", "U4", "photo", ["public_rejected"], status="rejected") + + # Force the states set_visibility would refuse, to prove the query itself + # filters rather than relying on the writer being careful. + with sqlite3.connect(path) as c: + c.execute("UPDATE media SET visibility='public' WHERE file_unique_id IN ('U2','U3','U4')") + + syncable = [r["keywords"] for r in await db.list_syncable()] + assert syncable == ["public_approved"], f"leaked: {syncable}" + assert await db.count_public() == 1 + print(" only approved AND public media is ever syncable") + + +async def test_sharing_requires_approval(tmp: str): + path = os.path.join(tmp, "share_guard.db") + db = load_db(path) + await db.init_db() + + await db.add_media(111, "F1", "U1", "photo", ["waiting"], status="pending") + pending = (await db.search_media("", status="pending"))[0] + assert await db.set_visibility(pending["id"], "public") is False, \ + "unreviewed media must not be shareable" + assert await db.list_syncable() == [] + print(" media awaiting moderation cannot be marked shared") + + +async def test_unsharing_stops_future_syncs(tmp: str): + path = os.path.join(tmp, "unshare.db") + db = load_db(path) + await db.init_db() + + await db.add_media(111, "F1", "U1", "photo", ["kw"]) + item = (await db.search_media("kw"))[0] + assert await db.set_visibility(item["id"], "public") is True + assert len(await db.list_syncable()) == 1 + + assert await db.set_visibility(item["id"], "private") is True + assert await db.list_syncable() == [], "unsharing should remove it from the sync set" + print(" unsharing takes an item back out of the sync set") + + +async def test_vault_never_syncs(tmp: str): + path = os.path.join(tmp, "vault_sync.db") + db = load_db(path) + await db.init_db() + + await db.add_vault_item(111, 500, [1, 2], "a private post", ["personal"]) + # The vault lives in its own table, so the sync query cannot reach it at all. + assert await db.list_syncable() == [] + assert await db.count_public() == 0 + print(" vault posts have no path into the sync set") + + async def main(): sys.path.insert(0, os.path.dirname(os.path.abspath(__file__))) with tempfile.TemporaryDirectory() as tmp: @@ -175,7 +251,10 @@ async def main(): test_readd_keeps_id, test_fresh_and_idempotent, test_pending_is_hidden, test_queue_is_oldest_first, test_two_moderators_cannot_double_review, - test_rejected_stays_hidden): + test_rejected_stays_hidden, test_new_media_is_private, + test_only_public_and_approved_can_sync, + test_sharing_requires_approval, + test_unsharing_stops_future_syncs, test_vault_never_syncs): await test(tmp) print("\nAll database tests passed.")