Commit graph

57 commits

Author SHA1 Message Date
soup
0391629862
chore(license): update copyright year in headers (#1929)
* chore: update copyright year in license headers

* Revert "chore: update copyright year in license headers"

This reverts commit 3e58129c431b9a491089ce36b908f9bb6ba38ed3.

* chore: update copyright year in license headers

* fix: sort go imports

* fix: add missing license headers
2025-01-06 22:23:19 +01:00
kenstir
4009554d10
feat(filters): skip duplicates (#1711)
* feat(filters): skip duplicates

* fix: add interface instead of any

* fix(filters): tonullint

* feat(filters): skip dupes check month day

* chore: cleanup

* feat(db): set autoincrement id

* feat(filters): add repack and proper to dupe profile

* feat(filters): add default dupe profiles

* feat(duplicates): check audio and website

* feat(duplicates): update tests

* feat(duplicates): add toggles on addform

* feat(duplicates): fix sqlite upgrade path and initialize duplicate profiles

* feat(duplicates): simplify sqlite upgrade

avoiding temp table and unwieldy select.  Besides, FK constraints
are turned off anyway in #229.

* feat(duplicates): change CheckIsDuplicateRelease treatment of PROPER and REPACK

"Proper" and "Repack" are not parallel to the other conditions like "Title",
so they do not belong as dedup conditions.  "PROPER" means there was an issue in
the previous release, and so a PROPER is never a duplicate, even if it replaces
another PROPER.  Similarly, "REPACK" means there was an issue in the previous
release by that group, and so it is a duplicate only if we previously took a
release from a DIFFERENT group.

I have not removed Proper and Repack from the UI or the schema yet.

* feat(duplicates): update postgres schema to match sqlite

* feat(duplicates): fix web build errors

* feat(duplicates): fix postgres errors

* feat(filters): do leftjoin for duplicate profile

* fix(filters): partial update dupe profile

* go fmt `internal/domain/filter.go`

* feat(duplicates): restore straightforward logic for proper/repack

* feat(duplicates): remove mostly duplicate TV duplicate profiles

Having one profile seems the cleanest.  If somebody wants multiple
resolutions then they can add Resolution to the duplicate profile.
Tested this profile with both weekly episodic releases and daily
show releases.

* feat(release): add db indexes and sub_title

* feat(release): add IsDuplicate tests

* feat(release): update action handler

* feat(release): add more tests for skip duplicates

* feat(duplicates): check audio

* feat(duplicates): add more tests

* feat(duplicates): match edition cut and more

* fix(duplicates): tests

* fix(duplicates): missing imports

* fix(duplicates): tests

* feat(duplicates): handle sub_title edition and language in ui

* fix(duplicates): tests

* feat(duplicates): check name against normalized hash

* fix(duplicates): tests

* chore: update .gitignore to ignore .pnpm-store

* fix: tests

* fix(filters): tests

* fix: bad conflict merge

* fix: update release type in test

* fix: use vendored hot-toast

* fix: release_test.go

* fix: rss_test.go

* feat(duplicates): improve title hashing for unique check

* feat(duplicates): further improve title hashing for unique check with lang

* feat(duplicates): fix tests

* feat(duplicates): add macros IsDuplicate and DuplicateProfile ID and name

* feat(duplicates): add normalized hash match option

* fix: headlessui-state prop warning

* fix(duplicates): add missing year in daily ep normalize

* fix(duplicates): check rejections len

---------

Co-authored-by: ze0s <ze0s@riseup.net>
2024-12-25 22:33:46 +01:00
Alexander Chernov
acef4ac624
feat(filters): RED and OPS fetch uploader from API (#1348)
* feat: added uploader when fetching torrent details on RED indexer

* revert

* tests

* refactor(filters): size and uploader api checks

* refactor(filters): fix test

* refactor(filters): add mutex to rejections

---------

Co-authored-by: Kyle Sanderson <kyle.leet@gmail.com>
Co-authored-by: ze0s <ze0s@riseup.net>
2024-12-17 20:39:09 +01:00
ze0s
f644b3a4d6
feat(filters): implement AnnounceType (#1837)
* feat(filters): implement AnnounceType

* fix: rss tests
2024-12-08 21:08:24 +01:00
Kyle Sanderson
4cc0f9cc83
feat(tests): add Parallelization where possible (#1823)
* feat(tests): add Parallelization where possible
2024-11-23 15:16:09 +01:00
ze0s
40fe3e9f54
fix(indexers): delete feed with indexer (#1810)
* fix(indexers): delete feed in one go

* fix(indexers): sort imports

* refactor(feeds): db methods for find
2024-11-06 20:48:46 +01:00
ze0s
ac6f62db99
fix(feeds): start feeds in background (#1718) 2024-09-12 12:13:05 +02:00
ze0s
abeb81eea9
fix(feeds): UNIT3D RSS size parsing (#1701)
* fix(feeds): size parsing for UNIT3D RSS feeds

* fix(feeds): imports
2024-09-02 16:39:40 +02:00
ze0s
bc0f4cc055
feat: add support for proxies to use with IRC and Indexers (#1421)
* feat: add support for proxies

* fix(http): release handler

* fix(migrations): define proxy early

* fix(migrations): pg proxy

* fix(proxy): list update delete

* fix(proxy): remove log and imports

* feat(irc): use proxy

* feat(irc): tests

* fix(web): update imports for ProxyForms.tsx

* fix(database): migration

* feat(proxy): test

* feat(proxy): validate proxy type

* feat(proxy): validate and test

* feat(proxy): improve validate and test

* feat(proxy): fix db schema

* feat(proxy): add db tests

* feat(proxy): handle http errors

* fix(http): imports

* feat(proxy): use proxy for indexer downloads

* feat(proxy): indexerforms select proxy

* feat(proxy): handle torrent download

* feat(proxy): skip if disabled

* feat(proxy): imports

* feat(proxy): implement in Feeds

* feat(proxy): update helper text indexer proxy

* feat(proxy): add internal cache
2024-09-02 11:10:45 +02:00
ze0s
59a10a910e
fix(feeds): implement staggered start (#1619) 2024-08-13 20:46:45 +02:00
Antoine
1530e9516d
fix(rss): use custom client for rss tests (#1557) 2024-05-19 14:55:51 +02:00
kenstir
4fceccd611
feat(filters): support daily shows (#1462)
* feat(database): Add month, day columns to release table

* feat(database): Add month, day columns to postgres release table

* feat(filters): support daily show format

* feat(filters): check smart episode daily

* fix(tests): rss

* feat(filters): add daily shows elements to form

* enhancement(web): minimize html in MoviesAndTV tab

* feat(filters): smart episode check proper and repack

* feat(filters): smart episode do not allow multiple latest

* feat(filters): smart episode check group with repack

* feat(filters): smart episode allow multiple current releases

---------

Co-authored-by: s0up4200 <soup@r4tio.dev>
Co-authored-by: ze0s <43699394+zze0s@users.noreply.github.com>
Co-authored-by: martylukyy <35452459+martylukyy@users.noreply.github.com>
2024-05-15 16:38:10 +02:00
ze0s
0016228d89
feat(indexers): add External Identifier to map with ARR indexers (#1534)
* feat(indexers): add External Identifier to map with ARR indexers

* fix: web build

* fix: tests

* feat: set identifier for manual processing
2024-05-04 12:37:01 +02:00
ze0s
56aa7dd5cb
fix(feeds): parse magnet URI from enclosure (#1514)
fix(feeds): parse magnet from enclosure
2024-04-19 14:05:28 +02:00
ze0s
3c3b47fa10
feat(macros): add IndexerName (#1511)
* feat(macros): add IndexerName

* fix: tests

* fix: tests
2024-04-16 17:35:17 +02:00
Kyle Sanderson
d86b2333a7
fix(feeds): always upgrade size from RSS description (#1458)
* fix(rss): add additional test for size

* always upgrade from desc
2024-03-19 17:06:49 +01:00
ze0s
5328078b32
fix(filters): RED and OPS lossless parsing and filtering (#1373)
* fix(filters): RED and OPS lossless parsing and filtering

* fix(filters): logscore and EP parsing

* fix(filters): tests

* fix(filters): tests

* feat(definitions): RED parse title variable

* feat(indexers): setup indexer to filter tests

* feat(indexers): tests and improve parsing

* feat(indexers): improve tests
2024-01-28 22:03:25 +01:00
Kyle Sanderson
cdd91d27e5
feat(feeds): improve RSS size parsing (#1367)
* fix(feeds): Parse multiple sizes.

* refactor: Test_pullSizeFromDescription

* refactor: make test human readable

added helper function

* multi

* Agnewwwwww

* .

* humanize

* humanize

---------

Co-authored-by: soup <soup@r4tio.dev>
2024-01-27 12:52:11 -08:00
luckyboy
a86258aaa7
feat(filters): implement min and max seeders/leechers filtering for Torznab feeds (#1342)
* feat(filter):implement min and max seeders/leechers filtering

* chore: go fmt and reorder fields

---------

Co-authored-by: ze0s <ze0s@riseup.net>
2024-01-12 17:08:18 +01:00
ze0s
45b522abf8
chore: update license header year (#1332)
* chore: update license header year

* chore: update license header year tsx files

* chore: update license header
2024-01-01 16:21:02 +01:00
Kyle Sanderson
3234f0d919
refactor(http): implement shared transport and clients (#1288)
* fix(http): flip to a shared transport and clients

* nice threads

* that is terrible

* fake uri for magnet

* lazy locking

* why bother with r's

* flip magic params to struct

* refactor(http-clients): use separate clients with shared transport

* refactor(http-clients): add missing license header

* refactor(http-clients): defer and fix errors

---------

Co-authored-by: ze0s <ze0s@riseup.net>
2023-12-29 23:49:22 +01:00
soup
2bd1a68a94
feat(feeds): add force run (#1243)
* feat(feeds): add force run

* fix: simplify ForceRun

* add confirmation modal

* handle errors by using the test func

* require user input to run

* make sure to reschedule next job after forcerun

* refactor modal centering with grid

* refactor: Simplify startJob and forceRun logic

- Refactor `startJob` to accept a `runImmediately` flag. This flag controls whether the job should be run immediately or scheduled for later. This change simplifies the `ForceRun` function by allowing it to call `startJob` with `runImmediately` set to `true`.

- Remove redundant checks in `ForceRun` related to feed type. These checks are handled in `startJob`.

BREAKING CHANGE: The `startJob` function now requires a second argument, `runImmediately`. This change affects all calls to `startJob`.

* fix(web) Invalidate queries after forceRun

* refactor(feeds): init and test run

---------

Co-authored-by: ze0s <43699394+zze0s@users.noreply.github.com>
2023-11-18 21:54:53 +01:00
ze0s
7c91d34cb4
fix(feeds): feed cache key (#1230)
* fix(feeds): feed cache sometimes missing key

* fix(feeds): feed cache use link as key if no guid
2023-10-30 22:10:03 +01:00
ze0s
9793764905
feat(feeds): improve caching (#1191)
* feat(feeds): improve caching

* fix(feeds): put cache if not empty

* fix(feeds): reassign loop var

* fix(feeds): enable busy_timeout again

* fix(feeds): enable busy_timeout again
2023-10-21 17:03:52 +02:00
jolinarofmalkshur
dbb3ff3a3b
feat(feeds): improve file size parsing (#1162)
* In rss feeds, sometimes the only place the size is mentioned is in the
description field. If the size has not already been determined from
another source try to read it from there.

* Accept sizes with no space between value and unit of measure

* feat(feeds): get size from description add test

* fix(feeds): tests

---------

Co-authored-by: ze0s <ze0s@riseup.net>
2023-10-01 22:47:47 +02:00
ze0s
1bfbe38335
feat(macros): add SizeString DownloadURL and InfoURL (#1080)
* feat(macro): add new variables

* SizeString
* DownloadURL
* InfoURL

* feat(macro): add new test cases
2023-09-03 13:27:47 +02:00
ze0s
a6d789ee44
feat(feeds): show next run (#1074) 2023-09-02 23:06:21 +02:00
ze0s
6fd8626507
feat(feeds): add scheduled cleanup (#1073)
* refactor(feeds): make feed scheduling more robust

* feat(feeds): add daily cleanup job

* removes feed cache older than 30 days

* fix(feeds): fmt wrong type
2023-09-02 22:44:28 +02:00
ze0s
b6de7144e0
feat(feeds): clear feed cache (#1071) 2023-09-01 21:39:39 +02:00
ze0s
e5b4ded725
feat(filters): add support for feed description (#922)
* feat(filters): match description

* feat(filters): support description

* chore: remove match logic for description

* fix: update rss tests
2023-05-13 19:33:18 +02:00
ze0s
01a69c7582
fix(feeds): process RSS on initial run (#918) 2023-05-07 15:55:04 +02:00
ze0s
604c7896bd
chore: add LICENSE GPLv2-or-later (#897)
* chore: add LICENSE

* chore: add LICENSE to README
2023-05-01 16:21:59 +02:00
ze0s
81d494ec5c
fix(feeds): torznab newznab category parsing (#775) 2023-03-19 23:35:33 +01:00
Kyle Sanderson
5836fcee63
fix(feeds): implement max_age for torznab/newznab (#763)
* fix(nab): Implement max_age for torznab/newznab

* continue newznab

* continue torznab
2023-03-19 21:26:18 +01:00
Kyle Sanderson
13a74f7cc8
feat: add usenet support (#543)
* feat(autobrr): implement usenet support

* feat(sonarr): implement usenet support

* feat(radarr): implement usenet support

* feat(announce): implement usenet support

* announce: cast a line

* feat(release): prevent unknown protocol transfer

* release: lines for days.

* feat: add newznab and sabnzbd support

* feat: add category to sabnzbd

* feat(newznab): map categories

* feat(newznab): map categories

---------

Co-authored-by: ze0s <43699394+zze0s@users.noreply.github.com>
Co-authored-by: ze0s <ze0s@riseup.net>
2023-03-04 20:27:18 +01:00
ze0s
ca196f0bf1
feat(releases): support magnet links (#730)
* feat(releases): support magnet links

* feat(feeds): support magnet links

* feat(actions): log messages

* fix: component warning

* fix: check hasprefix instead of hassuffix for magnet

* feat(release): resolve magnet uri from link

* fix(actions): deluge use magnet uri

* fix(macros): add `MagnetURI` var

* fix(actions): run magnet resolving before macros

* feat(feeds): set download type on creation
2023-02-28 22:16:10 +01:00
ze0s
5607184522
fix(releases): parse isScene as Origins SCENE (#716)
* fix(releases): parse isScene as Origins SCENE

* fix(releases): remove isScene
2023-02-14 22:48:10 +01:00
Ignacio
9a47dbc588
fix(feeds): RSS improve torrent size parsing (#667)
* Fixes issue 636 - fail to parse torrent size on some RSS feeds when enclosure is a fixed value

* Pick the biggest size

* minor comment

* Updates release size when new parsed string size is bigger

- Sets default size of `Release` to zero (unparsed)
- When there are parsing errors, then it doesn't RESET the size to 0 (assuming a tracker providing some size that can't be parsed)
- Updates the size to the new size only if the parsed size is bigger than the previous.

* fix(feeds): ignore default 39399 enclosure size

---------

Co-authored-by: ze0s <43699394+zze0s@users.noreply.github.com>
2023-02-05 19:18:17 +01:00
Michel Heusschen
9e0edc577f
enhancement(feeds): torznab freeleech percentage (#647) 2023-01-15 18:17:51 +01:00
ze0s
5c402b6d6c
feat(filters): support Language filtering (#632)
* feat(filters): add support for language

* feat(filters): add db migrations and repo

* feat(filters): fix failing tests

* feat(filters): fix failing tests
2023-01-09 00:33:08 +01:00
ze0s
fd67a7b24e
feat(feeds): view latest RSS and Torznab feed (#609)
feat(feeds): view latest run
2023-01-02 23:00:11 +01:00
Kyle Sanderson
6295239985
fix(rss): ignore max_age for old feed items (#527)
Feed dates are not always initialized, which defaults (if present) to epoch on some feed items. When this happens the results can be unpredictable, and not expected for what users are intending for. As such to have sane defaults, we ignore the date if it's before April 1st 1970, because nothing really happened before this date.
2022-11-07 17:32:49 +01:00
ze0s
e2bb14afa4
feat(feeds): improve RSS (#502)
* feat(feeds): improve rss

* save last_run time
* remove interval check
* refactor feed job keys
* add rss test
* add max_age check

* feat(feeds): rss basic freeleech parsing

* feat(feeds): rss cookie support

* feat(feeds): db get max_age

* feat(feeds): update log messages

* feat(feeds): pass cookie to release for download

* feat(feeds): improve size parsing

* feat(feeds): improve datetime check
2022-10-18 18:51:10 +02:00
ze0s
b7d2161fdb
feat(feeds): torznab parse category and freeleech (#492)
* feat(feeds): torznab parse freeleech

* feat(feeds): torznab parse categories
2022-10-14 18:01:04 +02:00
paperclip-go-brr
72be86a34f
feat(feed): Configurable request timeout (#456)
* feat(feed): Add field for setting request timeout

* fix: missing type in interface

* feat: add postgres migration and column to base schema
2022-10-04 17:33:35 +02:00
Kyle Sanderson
47eaeaa635
feat(rss): implement relative URL support (#484)
* feat(rss): implement relative URL support

* bp had some words

* feat(gh): uplift golang to 1.19.1

* feat(docker): uplift golang to 1.19

* feat(docker): uplift to go 1.19

* cleanup

* chore: update go version in go.mod

* feat: refactor and add test feed process item

* unescape query params after join
* break out logic into separate method
* add tests
2022-10-04 12:59:29 +02:00
ze0s
b50688159e
feat(feeds): add generic RSS support (#410)
* feat(feeds): add generic rss support

* feat(feeds/web): add generic rss support

* implement rss downloading

* gosum + mod

* re-add size from Custom field.

* implement uploader + category

* sync

* remove double assignment (+torznab)

* didn't save the rss file >.>

* cleanup

* fixfeeds): create rss indexer

* fix(feeds): stop feed

* feat(feeds): support nexusphp rss enclosure link

* feat(feeds): check size for custom size

* fix(feeds): race condition and only stop enabled feeds

* fix(feeds): unify indexer implementation badge

Co-authored-by: Kyle Sanderson <kyle.leet@gmail.com>
2022-08-20 00:34:46 +02:00
Kyle Sanderson
bbfa5627aa
fix(scheduler): use time as opposed to cron fields (#390)
* fix(torznab): use time for interval.

* flip Schedule to be duration based

* typefix

* Update service.go

* clear err

* pull err package

* casting to the moon

* enforce a minimum of 15m

* lower minimum based on feedback from Unit3d
2022-08-01 23:24:14 +02:00
Ludvig Lundgren
7af6f438b3
fix(feeds): torznab use link to download (#363) 2022-07-20 22:47:08 +02:00
Ludvig Lundgren
ebba72ec1f
feat(feeds): torznab add test button (#347) 2022-07-10 15:54:56 +02:00