e21414b2294e7d010b7c3d0a34eb926df7fbc3e4
12 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
|
|
e21414b229 |
Drop the legacy chapter model and narrow last page read to an Int
eu.kanade.tachiyomi.data.database.models.Chapter predates SQLDelight. It survived because it implements SChapter, so ReaderChapter could hand the same object to the database and to the source. ReaderChapter now holds the domain Chapter and converts at the three points that actually need a source model: getPageList, getFormat and getChapterUrl. The domain model is immutable, so the reader replaces the chapter rather than writing through it, behind an update function so the field cannot be reassigned from outside. Equality stays on the chapter it was loaded with, which keeps the identity comparisons in the viewers stable while progress is recorded against it. updateChapterProgress no longer mutates the chapter and reads the values straight back out to build the update it persists; it computes them once and passes them. Marking a chapter read moves out of the function named for completing it and into the same place the page is recorded. Removing the interface also drops the nullable id it carried, along with three non-null assertions and a Float to Double conversion that the compiler had been warning about. A page position is an index into a chapter's pages, so it cannot exceed Int range, but SQLite has one integer type and the column came back as a Long. That width reached the domain model and then every caller either carried it or converted, and the viewers had to narrow it again to index a list with. Declaring the column INTEGER AS Int moves the narrowing into a column adapter, so it happens once where the value stops being a database number. No migration: SQLite still stores an INTEGER, only the type SQLDelight generates changes. Backups are the exception and keep converting, because the field is on the wire as int64 and has to stay that way. That is also the only place the value arrives from outside the app, where a malformed file could carry something that does not fit and truncate to a negative index, so it is clamped rather than cast. |
||
|
|
2d1d2e4cab |
Rework database queries for the new schema (#4022)
* query - category Key every single-row statement on :id and list columns on one line. A lookup by primary key no longer needs LIMIT 1, and resetting every category's flags takes a value rather than a nullable nobody passed. Assisted-by: Claude:claude-opus-5 * query - chapter List columns in the table's order and name bind and mapper parameters after the columns they fill. Inserts pass their arguments by name, since reordering the columns put the upload and fetch dates side by side with the same type. With (manga_id, remote_url) unique, looking a chapter up by them no longer needs LIMIT 1. Available scanlators come back distinct instead of one row per chapter, which is all the only caller wanted. insert was insertReturningId without the RETURNING, so restore uses the latter and the former goes, along with the unused getChapterByUrl. Inserting skips a chapter that a sync running alongside stored first, rather than failing the whole batch and losing every new chapter of that update. Assisted-by: Claude:claude-opus-5 * query - excluded_scanlator An entry excludes a scanlator at most once now. Inserting one that is already excluded does nothing, so restoring a backup that lists one twice drops the repeat instead of failing the entry, and restore no longer reads the stored exclusions first to skip them itself. Saving an unchanged selection no longer runs an empty delete, which still made every chapter query run again. Assisted-by: Claude:claude-opus-5 * query - extension_store Take the upsert's new values from excluded and drop a WHERE its conflict target already implies. That also retires a `signing_key =: signingKey` that only parsed by accident. Assisted-by: Claude:claude-opus-5 * query - history Restoring an entry's history looked up the history row and, failing that, the chapter for every item in the backup, one query each. It now reads the entry's chapters and history once and matches them in memory, which leaves getHistoryByChapterUrlAndMangaId without callers. Creating a backup looked up the chapter of every history row one at a time and failed on a row whose chapter was gone. It reads the entry's chapters once and leaves such a row out. The upsert takes its new values from excluded, drops a trailing WHERE its conflict target already pins, and names its parameters after the columns they fill. Assisted-by: Claude:claude-opus-5 * query - manga List columns in the table's order and name bind and mapper parameters after the columns they fill, forwarding mapper arguments by name. With (source_id, remote_url) unique, looking an entry up by them no longer needs LIMIT 1, and fetching a network entry becomes a single upsert that refreshes an existing row only while it is outside the library, instead of an insert guarded by NOT EXISTS followed by an update of the same row. "Is anything read" checks use EXISTS rather than IN over a DISTINCT subquery, category membership drops a coalesce on a column that is never null, and the unused getAllManga goes. Applying the default chapter flags to the library is one transaction instead of one per entry, and restore checks backup entries against the stored urls as a set rather than a list. Assisted-by: Claude:claude-opus-5 * query - manga_category An entry belongs to a category at most once now. Inserting a pairing that already exists does nothing, so setting an entry's categories from a list that names one twice no longer fails, which restore could produce when two backup categories resolve to the same local one. Assisted-by: Claude:claude-opus-5 * query - manga_track List columns in the table's order and name bind and mapper parameters after the columns they fill. That retires the sync-era :syncId, :mediaId, :totalChapter and :trackingUrl. Assisted-by: Claude:claude-opus-5 * query - source List columns in the table's order and name the upsert's parameters after its columns. The upsert takes the conflicting row's values from excluded instead of rebinding them, and drops a WHERE its conflict target already implies. The call passes its two string arguments by name, since swapping them would still compile. Assisted-by: Claude:claude-opus-5 * query - historyView The latest read entry is read from the tables through the index on history's read time instead of aggregating the whole view for one row: ~50ms to under 1ms on a 114k chapter library. Assisted-by: Claude:claude-opus-5 * query - libraryView List the columns like the other queries. Assisted-by: Claude:claude-opus-5 * query - updatesView Drop a coalesce on category_id, which is never null, from the queries over updatesView, and the unused getRecentUpdates. The view definitions themselves are unchanged, so no migration is needed. The read and unread lookups become separate queries with the status written into each. SQLite only uses a partial index when the query's own condition matches it, which a bound parameter never does, so the unread lookup the widget follows walked nearly the whole fetch date index, ~60ms each time on a 114k chapter library and after every chapter or history write. With the status written in, it's served by the partial index over unread chapters, ~8ms. Assisted-by: Claude:claude-opus-5 * Query browse entries only while they are shown Each loaded entry kept a live query for as long as the screen existed, so every write to manga re-ran all of them. An entry now starts from the row the listing stored and queries only while shown. Assisted-by: Claude:claude-opus-5 * Read library selection categories from memory The change category dialog and category updates read each selected entry's categories from the loaded library instead of querying them one at a time. Assisted-by: Claude:claude-opus-5 |
||
|
|
93cc075116 |
Tweak database table and view schema (#3805)
* Select explicit columns and name same-typed arguments SQLDelight expands SELECT * using the table's column order and binds mappers to it by position, so reordering columns would silently swap mapper arguments. Queries list their columns instead. Calls whose arguments share a type pass them by name, so swapping two still fails to compile. The generated code is unchanged apart from the SQL text. * table - manga Rebuild `mangas` as `manga` with a UNIQUE(source_id, remote_url) constraint, `remote_`/`user_`/`state_` column prefixes and `_id` -> `id`. `remote_` is what the source reported, `user_` is what the user set, `state_` is local bookkeeping that is neither; `source_id` keeps its name because it identifies the source rather than carrying its data. Entries sharing a source and url are merged rather than dropped. The copies are ranked with the ones in the library first, since that is the entry the user has, then by age. The first one stays, takes each detail it is missing from the first copy that has it, and absorbs the children of the rest. The duplicate chapters that repointing creates are merged too, read state and history included. The child tables keep pointing at the table this replaces until their own step rebuilds them. Foreign keys are off for the whole migration, so nothing reads those clauses in between, and copying every child table here just to rewrite one clause would double the work the migration does. For the same reason the children resolve the surviving entry while they are copied rather than being rewritten here first. Merging entries is also what creates duplicate chapters and duplicate tracks, but each of those is resolved in the step that rebuilds its own table, where the constraint that makes the collision matter is introduced. Queries are only retargeted at the new names; using the new structure is left to later changes. Assisted-by: Claude:claude-opus-5 * table - chapter Rebuild `chapters` as `chapter` with a UNIQUE(manga_id, remote_url) constraint, `remote_`/`user_`/`state_` column prefixes and `_id` -> `id`. The entries that merged in the previous step brought their chapters with them, so the chapters move onto the surviving entry here and the duplicates that creates are merged: the copy with reading history is kept, the oldest otherwise, and the read state and history of the others are folded into it. It happens here rather than with the merge because this is where the constraint that rejects the duplicates is introduced. `chapters_manga_id_index`, `chapters_unread_by_manga_index` and `idx_chapters_url` go away: the new unique constraint indexes `manga_id`, and the only url-only lookup has no callers. The unread index served the updates view's read filter, but that view sorts by fetch date and stops at a limit, which an index on `state_date_fetch` serves better with or without the read filter, so that index takes its place. A second, partial one covers only the unread chapters' fetch dates: most chapters in a library are read, so finding the unread ones through the full index means walking nearly all of it. Queries are only retargeted at the new names; using the new structure is left to later changes. Assisted-by: Claude:claude-opus-5 * table - category Rebuild `categories` as `category` with `_id` -> `id` and `sort` -> `order`, the name it already had everywhere above the database. `order` gains a UNIQUE constraint, since two categories sharing a position was never meaningful. `sort` was never unique, and reordering wrote colliding values often enough that it cannot simply be copied under the constraint. The migration renumbers the rows from 0 in the order they were displayed, ties broken by id, and keeps the system category at -1. `mangas_categories` is rebuilt in the migration only to point its foreign key at the new table; it gets its real rebuild in its own step. Queries are only retargeted at the new names; using the new structure is left to later changes. Assisted-by: Claude:claude-opus-5 * table - manga_category Rebuild `mangas_categories` as `manga_category`. The surrogate `_id` went unused, so the pair becomes the identity with a UNIQUE(manga_id, category_id) constraint; duplicate memberships were never meaningful. `idx_mangas_categories_manga_id` goes away because the unique constraint already indexes `manga_id` as its leading column. Entries merged in the manga step bring their memberships along. When the merged entry is in the library, only the copies that were in it decide its categories, since a copy outside the library can still hold the categories it had before it was removed. Queries are only retargeted at the new names; using the new structure is left to later changes. Assisted-by: Claude:claude-opus-5 * table - history Rebuild `history` with `_id` -> `id`, `last_read` -> `read_at` and `time_read` -> `read_duration`; the old names read as booleans rather than the timestamp and duration they hold. `history_history_chapter_id_index` goes away because `chapter_id` is already UNIQUE and therefore indexed. A row also names the entry it belongs to now, not just the chapter. Reaching the entry meant joining through the chapter every time, and the entry is the thing history is read by. It is filled from the chapter it hangs off, and keyed to both: the entry first, then the chapter. A row whose chapter is already gone cannot name an entry and is dropped, which is what the foreign key would do to it once it is enforced again. The table is the only one here that keeps its name, so the original is renamed aside and the new one built under the real name. Building it beside the original and renaming it into place makes SQLite rewrite the stored definition, which would leave this table spelled differently from the same table on a fresh install. Queries are only retargeted at the new names; using the new structure is left to later changes. Assisted-by: Claude:claude-opus-5 * table - excluded_scanlator Rebuild `excluded_scanlators` as `excluded_scanlator`, singular like the other tables. The pair becomes the identity with a UNIQUE(manga_id, scanlator) constraint; excluding the same scanlator twice for one entry was never meaningful. `excluded_scanlators_manga_id_index` goes away because the unique constraint already indexes `manga_id` as its leading column. Queries are only retargeted at the new names; using the new structure is left to later changes. Assisted-by: Claude:claude-opus-5 * table - manga_track Rebuild `manga_sync` as `manga_track`: the table holds tracker entries, and nothing about it is a sync. `_id` -> `id` and `sync_id` -> `tracker_id`, which is what it identifies. `idx_manga_sync_manga_id` goes away because UNIQUE(manga_id, tracker_id) already indexes `manga_id` as its leading column. The constraint drops ON CONFLICT REPLACE and aborts like every other one. Replacing deleted the row and inserted a new one under a new id, which lost anything keyed to the old id, such as a queued tracking update, and it hid any other write that collided. The insert now says what a conflict means: an existing track for the entry and tracker is updated in place. When entries merge, two rows for the same tracker can end up pointing at the survivor, so the row to keep is chosen first: one on a copy in the library, then the one furthest read, then the one on the copy ranked first, then the oldest. A track is taken whole rather than merged field by field, because two rows on the same tracker can point at different remote entries. Queries are only retargeted at the new names; using the new structure is left to later changes. Assisted-by: Claude:claude-opus-5 * table - source Rebuild `sources` as `source` with `_id` -> `id` and `lang` -> `language`, spelled out like every other column here. Queries are only retargeted at the new names; using the new structure is left to later changes. Assisted-by: Claude:claude-opus-5 * view - updates The view is the Kotlin-facing shape of a row, so its columns are named the way the callers read them; `last_page_read` and `datefetch` were the only two that weren't. The migration builds every view once at its end, so this updates that definition rather than dropping and recreating it again. Queries are only retargeted at the new names; using the new structure is left to later changes. Assisted-by: Claude:claude-opus-5 * view - library Bound the chapter and category aggregates to the library. Both subqueries group over every chapter and every category membership in the database, and only the outer query drops the entries that are not in the library, so an entry that was merely browsed still pays for its chapters every time the library is read. The entries the view keeps are the only ones whose aggregates it uses, so the same filter belongs inside them, where `manga_user_favorite_at_index` can serve it. The view returns the same rows either way. On a 560k chapter database holding 400 entries among 4400, reading the library goes from ~156ms to ~23ms. Assisted-by: Claude:claude-opus-5 * view - history The view is the Kotlin-facing shape of a row, so its columns are named the way the callers read them: `source` becomes `sourceId` and `state_cover_last_modified` loses the prefix it only carried because the column it reads has one. The migration builds every view once at its end, so this updates that definition rather than dropping and recreating it again. Queries are only retargeted at the new names; using the new structure is left to later changes. Assisted-by: Claude:claude-opus-5 |
||
|
|
b74e010bfe |
Order updates by fetch date in the queries themselves
The updates queries relied on the view's ORDER BY surviving into the query that reads from it, which SQLite doesn't promise. Each now orders by fetch date itself. |
||
|
|
986f46c09a |
Merge manga favorite related fields into favorite_at
Migration 16 replaces favorite, date_added and favorite_modified_at with one nullable favorite_at. A favorite whose date_added is 0 falls back to favorite_modified_at. Manga.favorite and dateAdded become getters derived from it, and restore merges it by taking the earliest non-zero time. |
||
|
|
1d8a2b05dd |
Add category filters for Updates tab (#3589)
* Add category filters for Updates tab Works just like Library Updates. Provides a tab in the Updates tab filter dialog to allow for including & excluding of updates belonging to a title from a given category. Excludes overpower Includes, as with Library Updates. A little helpful text informs users about this. * Changelog * [skip ci] Remove unused string Leftover from before I realized the Default category will always be available for in-/exclusion. * Make :includeEmpty/:excludeEmpty true Boolean params * "include" -> "included" Consistency with #3607. * Remove DISTINCT from getRecentUpdatesWithFilters Apparently not needed anymore (though it was required at some point). * [skip ci] Fix Changelog |
||
|
|
bbe9aa8561 |
Add Filters to Updates screen (#2851)
* Add Filters to Updates screen Behaves basically like the filters in the library: - Unread: Show/Don't show unread chapters - Downloaded: Show/Don't show downloaded chapters - Started: Show/Don't show chapters that have some progress but aren't fully Read - Bookmarked: Show/Don't show chapters that have been bookmarked Started behaves differently from its Library counterpart because the actual manga data is not available at this point in time and I thought calling getManga for each entry without caching would be a pretty bad idea. I have modelled this closely on the filter control flow in the Library, but I'm sure this can be simplified/adjusted in some way. * Move most filtering logic to SQL Unread, Started, and Bookmarked filters are now part of the SQL query. Download state cannot be filtered in the database so it remains in Kotlin. Because the Downloaded filter has to be run in Kotlin, the combine flow uses the preferences flow twice, once to get the SQL query params and once for the Kotlin filters (only Downloaded at this time). * Add "Hide excluded scanlators" to update filters Based on the work done in #1623 but integrated with the other filters in this PR. Added the user as a co-author for credit. Co-authored-by: Dani <17619547+shabnix@users.noreply.github.com> --------- Co-authored-by: Dani <17619547+shabnix@users.noreply.github.com> |
||
|
|
58b25d697f |
Improve handling of downloads for chapters with same metadata and optionally for OSes that don't support Unicode in filename (#2305)
Co-authored-by: jkim <jhskim@hotmail.com> Co-authored-by: fatotak <111342761+fatotak@users.noreply.github.com> Co-authored-by: MajorTanya <39014446+MajorTanya@users.noreply.github.com> Co-authored-by: AntsyLich <59261191+AntsyLich@users.noreply.github.com> |
||
|
|
87530f506e |
Limit amount of updates loaded for widget
Probably fixes #9868 |
||
|
|
4c65c2311e |
Limit updates to 250 most recent chapters
Still limits to things within the past 3 months though. Closes #9554 |
||
|
|
3634b52e3a |
Only show unread entries in widget
Closes #9083 |
||
|
|
823749fc1e |
Move SQLDelight to data module (#8954)
And use tachiyomi instead of eu.kanade.tachiyomi for package names in the module |