Add download cleaner and dry run (#58)
This commit is contained in:
@@ -61,7 +61,7 @@ This tool is actively developed and still a work in progress, so using the `late
|
|||||||
2. **Queue cleaner** will:
|
2. **Queue cleaner** will:
|
||||||
- Run every 5 minutes (or configured cron, or right after `content blocker`).
|
- Run every 5 minutes (or configured cron, or right after `content blocker`).
|
||||||
- Process all items in the *arr queue.
|
- Process all items in the *arr queue.
|
||||||
- Check each queue item if it is **stalled (download speed is 0)**, **stuck in matadata downloading** or **failed to be imported**.
|
- Check each queue item if it is **stalled (download speed is 0)**, **stuck in metadata downloading** or **failed to be imported**.
|
||||||
- If it is, the item receives a **strike** and will continue to accumulate strikes every time it meets any of these conditions.
|
- If it is, the item receives a **strike** and will continue to accumulate strikes every time it meets any of these conditions.
|
||||||
- Check each queue item if it meets one of the following condition in the download client:
|
- Check each queue item if it meets one of the following condition in the download client:
|
||||||
- **Marked as completed, but 0 bytes have been downloaded** (due to files being blocked by qBittorrent or the **content blocker**).
|
- **Marked as completed, but 0 bytes have been downloaded** (due to files being blocked by qBittorrent or the **content blocker**).
|
||||||
@@ -71,6 +71,9 @@ This tool is actively developed and still a work in progress, so using the `late
|
|||||||
- It will be removed from the *arr's queue and blocked.
|
- It will be removed from the *arr's queue and blocked.
|
||||||
- It will be deleted from the download client.
|
- It will be deleted from the download client.
|
||||||
- A new search will be triggered for the *arr item.
|
- A new search will be triggered for the *arr item.
|
||||||
|
3. **Download cleaner** will:
|
||||||
|
- Run every hour (or configured cron).
|
||||||
|
- Automatically clean up downloads that have been seeding for a certain amount of time.
|
||||||
|
|
||||||
# Setup
|
# Setup
|
||||||
|
|
||||||
@@ -114,6 +117,8 @@ services:
|
|||||||
volumes:
|
volumes:
|
||||||
- ./cleanuperr/logs:/var/logs
|
- ./cleanuperr/logs:/var/logs
|
||||||
environment:
|
environment:
|
||||||
|
- DRY_RUN=false
|
||||||
|
|
||||||
- LOGGING__LOGLEVEL=Information
|
- LOGGING__LOGLEVEL=Information
|
||||||
- LOGGING__FILE__ENABLED=false
|
- LOGGING__FILE__ENABLED=false
|
||||||
- LOGGING__FILE__PATH=/var/logs/
|
- LOGGING__FILE__PATH=/var/logs/
|
||||||
@@ -121,6 +126,7 @@ services:
|
|||||||
|
|
||||||
- TRIGGERS__QUEUECLEANER=0 0/5 * * * ?
|
- TRIGGERS__QUEUECLEANER=0 0/5 * * * ?
|
||||||
- TRIGGERS__CONTENTBLOCKER=0 0/5 * * * ?
|
- TRIGGERS__CONTENTBLOCKER=0 0/5 * * * ?
|
||||||
|
- TRIGGERS__DOWNLOADCLEANER=0 0 * * * ?
|
||||||
|
|
||||||
- QUEUECLEANER__ENABLED=true
|
- QUEUECLEANER__ENABLED=true
|
||||||
- QUEUECLEANER__RUNSEQUENTIALLY=true
|
- QUEUECLEANER__RUNSEQUENTIALLY=true
|
||||||
@@ -138,6 +144,17 @@ services:
|
|||||||
- CONTENTBLOCKER__IGNORE_PRIVATE=false
|
- CONTENTBLOCKER__IGNORE_PRIVATE=false
|
||||||
- CONTENTBLOCKER__DELETE_PRIVATE=false
|
- CONTENTBLOCKER__DELETE_PRIVATE=false
|
||||||
|
|
||||||
|
- DOWNLOADCLEANER__ENABLED=true
|
||||||
|
- DOWNLOADCLEANER__DELETE_PRIVATE=false
|
||||||
|
- DOWNLOADCLEANER__CATEGORIES__0__NAME=tv-sonarr
|
||||||
|
- DOWNLOADCLEANER__CATEGORIES__0__MAX_RATIO=-1
|
||||||
|
- DOWNLOADCLEANER__CATEGORIES__0__MIN_SEED_TIME=0
|
||||||
|
- DOWNLOADCLEANER__CATEGORIES__0__MAX_SEED_TIME=240
|
||||||
|
- DOWNLOADCLEANER__CATEGORIES__1__NAME=radarr
|
||||||
|
- DOWNLOADCLEANER__CATEGORIES__1__MAX_RATIO=-1
|
||||||
|
- DOWNLOADCLEANER__CATEGORIES__1__MIN_SEED_TIME=0
|
||||||
|
- DOWNLOADCLEANER__CATEGORIES__1__MAX_SEED_TIME=240
|
||||||
|
|
||||||
- DOWNLOAD_CLIENT=none
|
- DOWNLOAD_CLIENT=none
|
||||||
# OR
|
# OR
|
||||||
# - DOWNLOAD_CLIENT=qBittorrent
|
# - DOWNLOAD_CLIENT=qBittorrent
|
||||||
@@ -179,139 +196,25 @@ services:
|
|||||||
- LIDARR__INSTANCES__1__URL=http://radarr:8687
|
- LIDARR__INSTANCES__1__URL=http://radarr:8687
|
||||||
- LIDARR__INSTANCES__1__APIKEY=secret6
|
- LIDARR__INSTANCES__1__APIKEY=secret6
|
||||||
|
|
||||||
# - NOTIFIARR__ON_IMPORT_FAILED_STRIKE=false
|
- NOTIFIARR__ON_IMPORT_FAILED_STRIKE=true
|
||||||
# - NOTIFIARR__ON_STALLED_STRIKE=false
|
- NOTIFIARR__ON_STALLED_STRIKE=true
|
||||||
# - NOTIFIARR__ON_QUEUE_ITEM_DELETE=false
|
- NOTIFIARR__ON_QUEUE_ITEM_DELETED=true
|
||||||
# - NOTIFIARR__API_KEY=notifiarr_secret
|
- NOTIFIARR__ON_DOWNLOAD_CLEANED=true
|
||||||
# - NOTIFIARR__CHANNEL_ID=discord_channel_id
|
- NOTIFIARR__API_KEY=notifiarr_secret
|
||||||
|
- NOTIFIARR__CHANNEL_ID=discord_channel_id
|
||||||
```
|
```
|
||||||
|
|
||||||
## Environment variables
|
## Environment variables
|
||||||
|
|
||||||
### General variables
|
Jump to:
|
||||||
<details>
|
- [General settings](variables.md#general-settings)
|
||||||
<summary>Click here</summary>
|
- [Queue Cleaner settings](variables.md#queue-cleaner-settings)
|
||||||
|
- [Content Blocker settings](variables.md#content-blocker-settings)
|
||||||
| Variable | Required | Description | Default value |
|
- [Download Cleaner settings](variables.md#download-cleaner-settings)
|
||||||
|---|---|---|---|
|
- [Download Client settings](variables.md#download-client-settings)
|
||||||
| LOGGING__LOGLEVEL | No | Can be `Verbose`, `Debug`, `Information`, `Warning`, `Error` or `Fatal`. | `Information` |
|
- [Arr settings](variables.md#arr-settings)
|
||||||
| LOGGING__FILE__ENABLED | No | Enable or disable logging to file. | false |
|
- [Notification settings](variables.md#notification-settings)
|
||||||
| LOGGING__FILE__PATH | No | Directory where to save the log files. | empty |
|
- [Advanced settings](variables.md#advanced-settings)
|
||||||
| LOGGING__ENHANCED | No | Enhance logs whenever possible.<br>A more detailed description is provided. [here](variables.md#LOGGING__ENHANCED) | true |
|
|
||||||
|||||
|
|
||||||
| TRIGGERS__QUEUECLEANER | Yes if queue cleaner is enabled | - [Quartz cron trigger](https://www.quartz-scheduler.org/documentation/quartz-2.3.0/tutorials/crontrigger.html).<br>- Can be a max of 6h interval.<br>- **Is ignored if `QUEUECLEANER__RUNSEQUENTIALLY=true` and `CONTENTBLOCKER__ENABLED=true`**. | 0 0/5 * * * ? |
|
|
||||||
| TRIGGERS__CONTENTBLOCKER | Yes if content blocker is enabled | - [Quartz cron trigger](https://www.quartz-scheduler.org/documentation/quartz-2.3.0/tutorials/crontrigger.html).<br>- Can be a max of 6h interval. | 0 0/5 * * * ? |
|
|
||||||
|||||
|
|
||||||
| QUEUECLEANER__ENABLED | No | Enable or disable the queue cleaner. | true |
|
|
||||||
| QUEUECLEANER__RUNSEQUENTIALLY | No | If set to true, the queue cleaner will run after the content blocker instead of running in parallel, streamlining the cleaning process. | true |
|
|
||||||
| QUEUECLEANER__IMPORT_FAILED_MAX_STRIKES | No | - After how many strikes should a failed import be removed.<br>- 0 means never. | 0 |
|
|
||||||
| QUEUECLEANER__IMPORT_FAILED_IGNORE_PRIVATE | No | Whether to ignore failed imports from private trackers. | false |
|
|
||||||
| QUEUECLEANER__IMPORT_FAILED_DELETE_PRIVATE | No | - Whether to delete failed imports of private downloads from the download client.<br>- Does not have any effect if `QUEUECLEANER__IMPORT_FAILED_IGNORE_PRIVATE` is `true`.<br>- **Set this to `true` if you don't care about seeding, ratio, H&R and potentially losing your tracker account.** | false |
|
|
||||||
| QUEUECLEANER__IMPORT_FAILED_IGNORE_PATTERNS__0 | No | - First pattern to look for when an import is failed.<br>- If the specified message pattern is found, the item is skipped. | empty |
|
|
||||||
| QUEUECLEANER__STALLED_MAX_STRIKES | No | - After how many strikes should a stalled download be removed.<br>- 0 means never. | 0 |
|
|
||||||
| QUEUECLEANER__STALLED_RESET_STRIKES_ON_PROGRESS | No | Whether to remove strikes if any download progress was made since last checked. | false |
|
|
||||||
| QUEUECLEANER__STALLED_IGNORE_PRIVATE | No | Whether to ignore stalled downloads from private trackers. | false |
|
|
||||||
| QUEUECLEANER__STALLED_DELETE_PRIVATE | No | - Whether to delete stalled private downloads from the download client.<br>- Does not have any effect if `QUEUECLEANER__STALLED_IGNORE_PRIVATE` is `true`.<br>- **Set this to `true` if you don't care about seeding, ratio, H&R and potentially losing your tracker account.** | false |
|
|
||||||
|||||
|
|
||||||
| CONTENTBLOCKER__ENABLED | No | Enable or disable the content blocker. | false |
|
|
||||||
| CONTENTBLOCKER__IGNORE_PRIVATE | No | Whether to ignore downloads from private trackers. | false |
|
|
||||||
| CONTENTBLOCKER__DELETE_PRIVATE | No | - Whether to delete private downloads that have all files blocked from the download client.<br>- Does not have any effect if `CONTENTBLOCKER__IGNORE_PRIVATE` is `true`.<br>- **Set this to `true` if you don't care about seeding, ratio, H&R and potentially losing your tracker account.** | false |
|
|
||||||
</details>
|
|
||||||
|
|
||||||
### Download client variables
|
|
||||||
<details>
|
|
||||||
<summary>Click here</summary>
|
|
||||||
|
|
||||||
| Variable | Required | Description | Default value |
|
|
||||||
|---|---|---|---|
|
|
||||||
| DOWNLOAD_CLIENT | No | Download client that is used by *arrs<br>Can be `qbittorrent`, `deluge`, `transmission` or `none` | `none` |
|
|
||||||
| QBITTORRENT__URL | No | qBittorrent instance url | http://localhost:8112 |
|
|
||||||
| QBITTORRENT__USERNAME | No | qBittorrent user | empty |
|
|
||||||
| QBITTORRENT__PASSWORD | No | qBittorrent password | empty |
|
|
||||||
|||||
|
|
||||||
| DELUGE__URL | No | Deluge instance url | http://localhost:8080 |
|
|
||||||
| DELUGE__PASSWORD | No | Deluge password | empty |
|
|
||||||
|||||
|
|
||||||
| TRANSMISSION__URL | No | Transmission instance url | http://localhost:9091 |
|
|
||||||
| TRANSMISSION__USERNAME | No | Transmission user | empty |
|
|
||||||
| TRANSMISSION__PASSWORD | No | Transmission password | empty |
|
|
||||||
</details>
|
|
||||||
|
|
||||||
### Arr variables
|
|
||||||
<details>
|
|
||||||
<summary>Click here</summary>
|
|
||||||
|
|
||||||
| Variable | Required | Description | Default value |
|
|
||||||
|---|---|---|---|
|
|
||||||
| SONARR__ENABLED | No | Enable or disable Sonarr cleanup | false |
|
|
||||||
| SONARR__BLOCK__TYPE | No | Block type<br>Can be `blacklist` or `whitelist` | `blacklist` |
|
|
||||||
| SONARR__BLOCK__PATH | No | Path to the blocklist (local file or url)<br>Needs to be json compatible | empty |
|
|
||||||
| SONARR__SEARCHTYPE | No | What to search for after removing a queue item<br>Can be `Episode`, `Season` or `Series` | `Episode` |
|
|
||||||
| SONARR__INSTANCES__0__URL | No | First Sonarr instance url | http://localhost:8989 |
|
|
||||||
| SONARR__INSTANCES__0__APIKEY | No | First Sonarr instance API key | empty |
|
|
||||||
|||||
|
|
||||||
| RADARR__ENABLED | No | Enable or disable Radarr cleanup | false |
|
|
||||||
| RADARR__BLOCK__TYPE | No | Block type<br>Can be `blacklist` or `whitelist` | `blacklist` |
|
|
||||||
| RADARR__BLOCK__PATH | No | Path to the blocklist (local file or url)<br>Needs to be json compatible | empty |
|
|
||||||
| RADARR__INSTANCES__0__URL | No | First Radarr instance url | http://localhost:7878 |
|
|
||||||
| RADARR__INSTANCES__0__APIKEY | No | First Radarr instance API key | empty |
|
|
||||||
|||||
|
|
||||||
| LIDARR__ENABLED | No | Enable or disable LIDARR cleanup | false |
|
|
||||||
| LIDARR__BLOCK__TYPE | No | Block type<br>Can be `blacklist` or `whitelist` | `blacklist` |
|
|
||||||
| LIDARR__BLOCK__PATH | No | Path to the blocklist (local file or url)<br>Needs to be json compatible | empty |
|
|
||||||
| LIDARR__INSTANCES__0__URL | No | First LIDARR instance url | http://localhost:8686 |
|
|
||||||
| LIDARR__INSTANCES__0__APIKEY | No | First LIDARR instance API key | empty |
|
|
||||||
</details>
|
|
||||||
|
|
||||||
### Notifications variables
|
|
||||||
<details>
|
|
||||||
<summary>Click here</summary>
|
|
||||||
|
|
||||||
| Variable | Required | Description | Default value |
|
|
||||||
|---|---|---|---|
|
|
||||||
| NOTIFIARR__ON_IMPORT_FAILED_STRIKE | No | Notify on failed import strike. | false |
|
|
||||||
| NOTIFIARR__ON_STALLED_STRIKE | No | Notify on stalled download strike. | false |
|
|
||||||
| NOTIFIARR__ON_QUEUE_ITEM_DELETE | No | Notify on deleting a queue item. | false |
|
|
||||||
| NOTIFIARR__API_KEY | No | Notifiarr API key.<br>Requires Notifiarr's `Passthrough` integration to work. | empty |
|
|
||||||
| NOTIFIARR__CHANNEL_ID | No | Discord channel id for notifications. | empty |
|
|
||||||
</details>
|
|
||||||
|
|
||||||
|
|
||||||
### Advanced variables
|
|
||||||
<details>
|
|
||||||
<summary>Click here</summary>
|
|
||||||
|
|
||||||
| Variable | Required | Description | Default value |
|
|
||||||
|---|---|---|---|
|
|
||||||
| HTTP_MAX_RETRIES | No | The number of times to retry a failed HTTP call (to *arrs, download clients etc.) | 0 |
|
|
||||||
| HTTP_TIMEOUT | No | The number of seconds to wait before failing an HTTP call (to *arrs, download clients etc.) | 100 |
|
|
||||||
</details>
|
|
||||||
|
|
||||||
#
|
|
||||||
|
|
||||||
> [!NOTE]
|
|
||||||
> 1. The queue cleaner and content blocker can be enabled or disabled separately, if you want to run only one of them.
|
|
||||||
> 2. Only one download client can be enabled at a time. If you have more than one download client, you should deploy multiple instances of cleanuperr.
|
|
||||||
> 3. The blocklists (blacklist/whitelist) should have a single pattern on each line and supports the following:
|
|
||||||
> ```
|
|
||||||
> *example // file name ends with "example"
|
|
||||||
> example* // file name starts with "example"
|
|
||||||
> *example* // file name has "example" in the name
|
|
||||||
> example // file name is exactly the word "example"
|
|
||||||
> regex:<ANY_REGEX> // regex that needs to be marked at the start of the line with "regex:"
|
|
||||||
> ```
|
|
||||||
> 4. Multiple Sonarr/Radarr/Lidarr instances can be specified using this format, where `<NUMBER>` starts from `0`:
|
|
||||||
> ```
|
|
||||||
> SONARR__INSTANCES__<NUMBER>__URL
|
|
||||||
> SONARR__INSTANCES__<NUMBER>__APIKEY
|
|
||||||
> ```
|
|
||||||
> 5. Multiple failed import patterns can be specified using this format, where `<NUMBER>` starts from 0:
|
|
||||||
> ```
|
|
||||||
> QUEUECLEANER__IMPORT_FAILED_IGNORE_PATTERNS__<NUMBER>
|
|
||||||
> ```
|
|
||||||
> 6. [This blacklist](https://raw.githubusercontent.com/flmorg/cleanuperr/refs/heads/main/blacklist) and [this whitelist](https://raw.githubusercontent.com/flmorg/cleanuperr/refs/heads/main/whitelist) can be used for Sonarr and Radarr, but they are not suitable for other *arrs.
|
|
||||||
|
|
||||||
#
|
|
||||||
|
|
||||||
### Binaries (if you're not using Docker)
|
### Binaries (if you're not using Docker)
|
||||||
|
|
||||||
@@ -328,9 +231,10 @@ Special thanks for inspiration go to:
|
|||||||
- [ThijmenGThN/swaparr](https://github.com/ThijmenGThN/swaparr)
|
- [ThijmenGThN/swaparr](https://github.com/ThijmenGThN/swaparr)
|
||||||
- [ManiMatter/decluttarr](https://github.com/ManiMatter/decluttarr)
|
- [ManiMatter/decluttarr](https://github.com/ManiMatter/decluttarr)
|
||||||
- [PaeyMoopy/sonarr-radarr-queue-cleaner](https://github.com/PaeyMoopy/sonarr-radarr-queue-cleaner)
|
- [PaeyMoopy/sonarr-radarr-queue-cleaner](https://github.com/PaeyMoopy/sonarr-radarr-queue-cleaner)
|
||||||
- [Sonarr](https://github.com/Sonarr/Sonarr) & [Radarr](https://github.com/Radarr/Radarr) for the logo
|
- [Sonarr](https://github.com/Sonarr/Sonarr) & [Radarr](https://github.com/Radarr/Radarr)
|
||||||
|
|
||||||
# Buy me a coffee
|
# Buy me a coffee
|
||||||
If I made your life just a tiny bit easier, consider buying me a coffee!
|
If I made your life just a tiny bit easier, consider buying me a coffee!
|
||||||
|
|
||||||
<a href="https://buymeacoffee.com/flaminel" target="_blank"><img src="https://www.buymeacoffee.com/assets/img/custom_images/orange_img.png" alt="Buy Me A Coffee" style="height: 41px !important;width: 174px !important;box-shadow: 0px 3px 2px 0px rgba(190, 190, 190, 0.5) !important;-webkit-box-shadow: 0px 3px 2px 0px rgba(190, 190, 190, 0.5) !important;" ></a>
|
<a href="https://buymeacoffee.com/flaminel" target="_blank"><img src="https://www.buymeacoffee.com/assets/img/custom_images/orange_img.png" alt="Buy Me A Coffee" style="height: 41px !important;width: 174px !important;box-shadow: 0px 3px 2px 0px rgba(190, 190, 190, 0.5) !important;-webkit-box-shadow: 0px 3px 2px 0px rgba(190, 190, 190, 0.5) !important;" ></a>
|
||||||
|
|
||||||
|
|||||||
@@ -0,0 +1,6 @@
|
|||||||
|
namespace Common.Attributes;
|
||||||
|
|
||||||
|
[AttributeUsage(AttributeTargets.Method, Inherited = true)]
|
||||||
|
public class DryRunSafeguardAttribute : Attribute
|
||||||
|
{
|
||||||
|
}
|
||||||
@@ -7,7 +7,7 @@
|
|||||||
</PropertyGroup>
|
</PropertyGroup>
|
||||||
|
|
||||||
<ItemGroup>
|
<ItemGroup>
|
||||||
<PackageReference Include="Microsoft.Extensions.Configuration.Abstractions" Version="9.0.0" />
|
<PackageReference Include="Microsoft.Extensions.Configuration.Abstractions" Version="9.0.2" />
|
||||||
<PackageReference Include="Serilog" Version="4.2.0" />
|
<PackageReference Include="Serilog" Version="4.2.0" />
|
||||||
</ItemGroup>
|
</ItemGroup>
|
||||||
|
|
||||||
|
|||||||
@@ -11,9 +11,9 @@ public abstract record ArrConfig
|
|||||||
public required List<ArrInstance> Instances { get; init; }
|
public required List<ArrInstance> Instances { get; init; }
|
||||||
}
|
}
|
||||||
|
|
||||||
public record Block
|
public readonly record struct Block
|
||||||
{
|
{
|
||||||
public BlocklistType Type { get; set; }
|
public BlocklistType Type { get; init; }
|
||||||
|
|
||||||
public string? Path { get; set; }
|
public string? Path { get; init; }
|
||||||
}
|
}
|
||||||
@@ -0,0 +1,45 @@
|
|||||||
|
using Common.Exceptions;
|
||||||
|
using Microsoft.Extensions.Configuration;
|
||||||
|
|
||||||
|
namespace Common.Configuration.DownloadCleaner;
|
||||||
|
|
||||||
|
public sealed record Category : IConfig
|
||||||
|
{
|
||||||
|
public required string Name { get; init; }
|
||||||
|
|
||||||
|
/// <summary>
|
||||||
|
/// Max ratio before removing a download.
|
||||||
|
/// </summary>
|
||||||
|
[ConfigurationKeyName("MAX_RATIO")]
|
||||||
|
public required double MaxRatio { get; init; } = -1;
|
||||||
|
|
||||||
|
/// <summary>
|
||||||
|
/// Min number of hours to seed before removing a download, if the ratio has been met.
|
||||||
|
/// </summary>
|
||||||
|
[ConfigurationKeyName("MIN_SEED_TIME")]
|
||||||
|
public required double MinSeedTime { get; init; } = 0;
|
||||||
|
|
||||||
|
/// <summary>
|
||||||
|
/// Number of hours to seed before removing a download.
|
||||||
|
/// </summary>
|
||||||
|
[ConfigurationKeyName("MAX_SEED_TIME")]
|
||||||
|
public required double MaxSeedTime { get; init; } = -1;
|
||||||
|
|
||||||
|
public void Validate()
|
||||||
|
{
|
||||||
|
if (string.IsNullOrWhiteSpace(Name))
|
||||||
|
{
|
||||||
|
throw new ValidationException($"{nameof(Name)} can not be empty");
|
||||||
|
}
|
||||||
|
|
||||||
|
if (MaxRatio < 0 && MaxSeedTime < 0)
|
||||||
|
{
|
||||||
|
throw new ValidationException($"both {nameof(MaxRatio)} and {nameof(MaxSeedTime)} are disabled");
|
||||||
|
}
|
||||||
|
|
||||||
|
if (MinSeedTime < 0)
|
||||||
|
{
|
||||||
|
throw new ValidationException($"{nameof(MinSeedTime)} can not be negative");
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,36 @@
|
|||||||
|
using Common.Exceptions;
|
||||||
|
using Microsoft.Extensions.Configuration;
|
||||||
|
|
||||||
|
namespace Common.Configuration.DownloadCleaner;
|
||||||
|
|
||||||
|
public sealed record DownloadCleanerConfig : IJobConfig
|
||||||
|
{
|
||||||
|
public const string SectionName = "DownloadCleaner";
|
||||||
|
|
||||||
|
public bool Enabled { get; init; }
|
||||||
|
|
||||||
|
public List<Category>? Categories { get; init; }
|
||||||
|
|
||||||
|
[ConfigurationKeyName("DELETE_PRIVATE")]
|
||||||
|
public bool DeletePrivate { get; set; }
|
||||||
|
|
||||||
|
public void Validate()
|
||||||
|
{
|
||||||
|
if (!Enabled)
|
||||||
|
{
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
|
||||||
|
if (Categories?.Count is null or 0)
|
||||||
|
{
|
||||||
|
throw new ValidationException("no categories configured");
|
||||||
|
}
|
||||||
|
|
||||||
|
if (Categories?.GroupBy(x => x.Name).Any(x => x.Count() > 1) is true)
|
||||||
|
{
|
||||||
|
throw new ValidationException("duplicated categories found");
|
||||||
|
}
|
||||||
|
|
||||||
|
Categories?.ForEach(x => x.Validate());
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -1,4 +1,6 @@
|
|||||||
namespace Common.Configuration.DownloadClient;
|
using Common.Exceptions;
|
||||||
|
|
||||||
|
namespace Common.Configuration.DownloadClient;
|
||||||
|
|
||||||
public sealed record DelugeConfig : IConfig
|
public sealed record DelugeConfig : IConfig
|
||||||
{
|
{
|
||||||
@@ -12,7 +14,7 @@ public sealed record DelugeConfig : IConfig
|
|||||||
{
|
{
|
||||||
if (Url is null)
|
if (Url is null)
|
||||||
{
|
{
|
||||||
throw new ArgumentNullException(nameof(Url));
|
throw new ValidationException($"{nameof(Url)} is empty");
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
@@ -1,4 +1,6 @@
|
|||||||
namespace Common.Configuration.DownloadClient;
|
using Common.Exceptions;
|
||||||
|
|
||||||
|
namespace Common.Configuration.DownloadClient;
|
||||||
|
|
||||||
public sealed class QBitConfig : IConfig
|
public sealed class QBitConfig : IConfig
|
||||||
{
|
{
|
||||||
@@ -14,7 +16,7 @@ public sealed class QBitConfig : IConfig
|
|||||||
{
|
{
|
||||||
if (Url is null)
|
if (Url is null)
|
||||||
{
|
{
|
||||||
throw new ArgumentNullException(nameof(Url));
|
throw new ValidationException($"{nameof(Url)} is empty");
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
@@ -1,4 +1,6 @@
|
|||||||
namespace Common.Configuration.DownloadClient;
|
using Common.Exceptions;
|
||||||
|
|
||||||
|
namespace Common.Configuration.DownloadClient;
|
||||||
|
|
||||||
public record TransmissionConfig : IConfig
|
public record TransmissionConfig : IConfig
|
||||||
{
|
{
|
||||||
@@ -14,7 +16,7 @@ public record TransmissionConfig : IConfig
|
|||||||
{
|
{
|
||||||
if (Url is null)
|
if (Url is null)
|
||||||
{
|
{
|
||||||
throw new ArgumentNullException(nameof(Url));
|
throw new ValidationException($"{nameof(Url)} is empty");
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
@@ -0,0 +1,9 @@
|
|||||||
|
using Microsoft.Extensions.Configuration;
|
||||||
|
|
||||||
|
namespace Common.Configuration.General;
|
||||||
|
|
||||||
|
public sealed record DryRunConfig
|
||||||
|
{
|
||||||
|
[ConfigurationKeyName("DRY_RUN")]
|
||||||
|
public bool IsDryRun { get; init; }
|
||||||
|
}
|
||||||
@@ -1,8 +1,9 @@
|
|||||||
using Microsoft.Extensions.Configuration;
|
using Common.Exceptions;
|
||||||
|
using Microsoft.Extensions.Configuration;
|
||||||
|
|
||||||
namespace Common.Configuration.General;
|
namespace Common.Configuration.General;
|
||||||
|
|
||||||
public class HttpConfig : IConfig
|
public sealed record HttpConfig : IConfig
|
||||||
{
|
{
|
||||||
[ConfigurationKeyName("HTTP_MAX_RETRIES")]
|
[ConfigurationKeyName("HTTP_MAX_RETRIES")]
|
||||||
public ushort MaxRetries { get; init; }
|
public ushort MaxRetries { get; init; }
|
||||||
@@ -14,7 +15,7 @@ public class HttpConfig : IConfig
|
|||||||
{
|
{
|
||||||
if (Timeout is 0)
|
if (Timeout is 0)
|
||||||
{
|
{
|
||||||
throw new ArgumentException("HTTP_TIMEOUT must be greater than 0");
|
throw new ValidationException("HTTP_TIMEOUT must be greater than 0");
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
@@ -7,4 +7,6 @@ public sealed class TriggersConfig
|
|||||||
public required string QueueCleaner { get; init; }
|
public required string QueueCleaner { get; init; }
|
||||||
|
|
||||||
public required string ContentBlocker { get; init; }
|
public required string ContentBlocker { get; init; }
|
||||||
|
|
||||||
|
public required string DownloadCleaner { get; init; }
|
||||||
}
|
}
|
||||||
@@ -10,10 +10,13 @@ public abstract record NotificationConfig
|
|||||||
[ConfigurationKeyName("ON_STALLED_STRIKE")]
|
[ConfigurationKeyName("ON_STALLED_STRIKE")]
|
||||||
public bool OnStalledStrike { get; init; }
|
public bool OnStalledStrike { get; init; }
|
||||||
|
|
||||||
[ConfigurationKeyName("ON_QUEUE_ITEM_DELETE")]
|
[ConfigurationKeyName("ON_QUEUE_ITEM_DELETED")]
|
||||||
public bool OnQueueItemDelete { get; init; }
|
public bool OnQueueItemDeleted { get; init; }
|
||||||
|
|
||||||
public bool IsEnabled => OnImportFailedStrike || OnStalledStrike || OnQueueItemDelete;
|
[ConfigurationKeyName("ON_DOWNLOAD_CLEANED")]
|
||||||
|
public bool OnDownloadCleaned { get; init; }
|
||||||
|
|
||||||
|
public bool IsEnabled => OnImportFailedStrike || OnStalledStrike || OnQueueItemDeleted || OnDownloadCleaned;
|
||||||
|
|
||||||
public abstract bool IsValid();
|
public abstract bool IsValid();
|
||||||
}
|
}
|
||||||
@@ -0,0 +1,12 @@
|
|||||||
|
namespace Common.Exceptions;
|
||||||
|
|
||||||
|
public sealed class ValidationException : Exception
|
||||||
|
{
|
||||||
|
public ValidationException()
|
||||||
|
{
|
||||||
|
}
|
||||||
|
|
||||||
|
public ValidationException(string message) : base(message)
|
||||||
|
{
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,8 @@
|
|||||||
|
namespace Domain.Enums;
|
||||||
|
|
||||||
|
public enum CleanReason
|
||||||
|
{
|
||||||
|
None,
|
||||||
|
MaxRatioReached,
|
||||||
|
MaxSeedTimeReached,
|
||||||
|
}
|
||||||
@@ -16,4 +16,11 @@ public sealed record TorrentStatus
|
|||||||
|
|
||||||
[JsonProperty("total_done")]
|
[JsonProperty("total_done")]
|
||||||
public long TotalDone { get; init; }
|
public long TotalDone { get; init; }
|
||||||
|
|
||||||
|
public string? Label { get; init; }
|
||||||
|
|
||||||
|
[JsonProperty("seeding_time")]
|
||||||
|
public long SeedingTime { get; init; }
|
||||||
|
|
||||||
|
public float Ratio { get; init; }
|
||||||
}
|
}
|
||||||
@@ -1,6 +1,8 @@
|
|||||||
using Common.Configuration.Arr;
|
using Common.Configuration.Arr;
|
||||||
using Common.Configuration.ContentBlocker;
|
using Common.Configuration.ContentBlocker;
|
||||||
|
using Common.Configuration.DownloadCleaner;
|
||||||
using Common.Configuration.DownloadClient;
|
using Common.Configuration.DownloadClient;
|
||||||
|
using Common.Configuration.General;
|
||||||
using Common.Configuration.Logging;
|
using Common.Configuration.Logging;
|
||||||
using Common.Configuration.QueueCleaner;
|
using Common.Configuration.QueueCleaner;
|
||||||
|
|
||||||
@@ -10,8 +12,10 @@ public static class ConfigurationDI
|
|||||||
{
|
{
|
||||||
public static IServiceCollection AddConfiguration(this IServiceCollection services, IConfiguration configuration) =>
|
public static IServiceCollection AddConfiguration(this IServiceCollection services, IConfiguration configuration) =>
|
||||||
services
|
services
|
||||||
|
.Configure<DryRunConfig>(configuration)
|
||||||
.Configure<QueueCleanerConfig>(configuration.GetSection(QueueCleanerConfig.SectionName))
|
.Configure<QueueCleanerConfig>(configuration.GetSection(QueueCleanerConfig.SectionName))
|
||||||
.Configure<ContentBlockerConfig>(configuration.GetSection(ContentBlockerConfig.SectionName))
|
.Configure<ContentBlockerConfig>(configuration.GetSection(ContentBlockerConfig.SectionName))
|
||||||
|
.Configure<DownloadCleanerConfig>(configuration.GetSection(DownloadCleanerConfig.SectionName))
|
||||||
.Configure<DownloadClientConfig>(configuration)
|
.Configure<DownloadClientConfig>(configuration)
|
||||||
.Configure<QBitConfig>(configuration.GetSection(QBitConfig.SectionName))
|
.Configure<QBitConfig>(configuration.GetSection(QBitConfig.SectionName))
|
||||||
.Configure<DelugeConfig>(configuration.GetSection(DelugeConfig.SectionName))
|
.Configure<DelugeConfig>(configuration.GetSection(DelugeConfig.SectionName))
|
||||||
|
|||||||
@@ -1,6 +1,7 @@
|
|||||||
using Common.Configuration.Logging;
|
using Common.Configuration.Logging;
|
||||||
using Domain.Enums;
|
using Domain.Enums;
|
||||||
using Infrastructure.Verticals.ContentBlocker;
|
using Infrastructure.Verticals.ContentBlocker;
|
||||||
|
using Infrastructure.Verticals.DownloadCleaner;
|
||||||
using Infrastructure.Verticals.QueueCleaner;
|
using Infrastructure.Verticals.QueueCleaner;
|
||||||
using Serilog;
|
using Serilog;
|
||||||
using Serilog.Events;
|
using Serilog.Events;
|
||||||
@@ -33,7 +34,7 @@ public static class LoggingDI
|
|||||||
const string consoleOutputTemplate = $"[{{@t:yyyy-MM-dd HH:mm:ss.fff}} {{@l:u3}}]{jobNameTemplate}{instanceNameTemplate} {{@m}}\n{{@x}}";
|
const string consoleOutputTemplate = $"[{{@t:yyyy-MM-dd HH:mm:ss.fff}} {{@l:u3}}]{jobNameTemplate}{instanceNameTemplate} {{@m}}\n{{@x}}";
|
||||||
const string fileOutputTemplate = $"{{@t:yyyy-MM-dd HH:mm:ss.fff zzz}} [{{@l:u3}}]{jobNameTemplate}{instanceNameTemplate} {{@m:lj}}\n{{@x}}";
|
const string fileOutputTemplate = $"{{@t:yyyy-MM-dd HH:mm:ss.fff zzz}} [{{@l:u3}}]{jobNameTemplate}{instanceNameTemplate} {{@m:lj}}\n{{@x}}";
|
||||||
LogEventLevel level = LogEventLevel.Information;
|
LogEventLevel level = LogEventLevel.Information;
|
||||||
List<string> names = [nameof(ContentBlocker), nameof(QueueCleaner)];
|
List<string> names = [nameof(ContentBlocker), nameof(QueueCleaner), nameof(DownloadCleaner)];
|
||||||
int jobPadding = names.Max(x => x.Length) + 2;
|
int jobPadding = names.Max(x => x.Length) + 2;
|
||||||
names = [InstanceType.Sonarr.ToString(), InstanceType.Radarr.ToString(), InstanceType.Lidarr.ToString()];
|
names = [InstanceType.Sonarr.ToString(), InstanceType.Radarr.ToString(), InstanceType.Lidarr.ToString()];
|
||||||
int arrPadding = names.Max(x => x.Length) + 2;
|
int arrPadding = names.Max(x => x.Length) + 2;
|
||||||
|
|||||||
@@ -1,6 +1,8 @@
|
|||||||
using System.Net;
|
using System.Net;
|
||||||
|
using Castle.DynamicProxy;
|
||||||
using Common.Configuration.General;
|
using Common.Configuration.General;
|
||||||
using Common.Helpers;
|
using Common.Helpers;
|
||||||
|
using Infrastructure.Interceptors;
|
||||||
using Infrastructure.Verticals.DownloadClient.Deluge;
|
using Infrastructure.Verticals.DownloadClient.Deluge;
|
||||||
using Infrastructure.Verticals.Notifications.Consumers;
|
using Infrastructure.Verticals.Notifications.Consumers;
|
||||||
using Infrastructure.Verticals.Notifications.Models;
|
using Infrastructure.Verticals.Notifications.Models;
|
||||||
@@ -25,7 +27,8 @@ public static class MainDI
|
|||||||
{
|
{
|
||||||
config.AddConsumer<NotificationConsumer<FailedImportStrikeNotification>>();
|
config.AddConsumer<NotificationConsumer<FailedImportStrikeNotification>>();
|
||||||
config.AddConsumer<NotificationConsumer<StalledStrikeNotification>>();
|
config.AddConsumer<NotificationConsumer<StalledStrikeNotification>>();
|
||||||
config.AddConsumer<NotificationConsumer<QueueItemDeleteNotification>>();
|
config.AddConsumer<NotificationConsumer<QueueItemDeletedNotification>>();
|
||||||
|
config.AddConsumer<NotificationConsumer<DownloadCleanedNotification>>();
|
||||||
|
|
||||||
config.UsingInMemory((context, cfg) =>
|
config.UsingInMemory((context, cfg) =>
|
||||||
{
|
{
|
||||||
@@ -33,12 +36,14 @@ public static class MainDI
|
|||||||
{
|
{
|
||||||
e.ConfigureConsumer<NotificationConsumer<FailedImportStrikeNotification>>(context);
|
e.ConfigureConsumer<NotificationConsumer<FailedImportStrikeNotification>>(context);
|
||||||
e.ConfigureConsumer<NotificationConsumer<StalledStrikeNotification>>(context);
|
e.ConfigureConsumer<NotificationConsumer<StalledStrikeNotification>>(context);
|
||||||
e.ConfigureConsumer<NotificationConsumer<QueueItemDeleteNotification>>(context);
|
e.ConfigureConsumer<NotificationConsumer<QueueItemDeletedNotification>>(context);
|
||||||
|
e.ConfigureConsumer<NotificationConsumer<DownloadCleanedNotification>>(context);
|
||||||
e.ConcurrentMessageLimit = 1;
|
e.ConcurrentMessageLimit = 1;
|
||||||
e.PrefetchCount = 1;
|
e.PrefetchCount = 1;
|
||||||
});
|
});
|
||||||
});
|
});
|
||||||
});
|
})
|
||||||
|
.AddDryRunInterceptor();
|
||||||
|
|
||||||
private static IServiceCollection AddHttpClients(this IServiceCollection services, IConfiguration configuration)
|
private static IServiceCollection AddHttpClients(this IServiceCollection services, IConfiguration configuration)
|
||||||
{
|
{
|
||||||
@@ -86,4 +91,31 @@ public static class MainDI
|
|||||||
.OrResult(response => !response.IsSuccessStatusCode && response.StatusCode != HttpStatusCode.Unauthorized)
|
.OrResult(response => !response.IsSuccessStatusCode && response.StatusCode != HttpStatusCode.Unauthorized)
|
||||||
.WaitAndRetryAsync(config.MaxRetries, retryAttempt => TimeSpan.FromSeconds(Math.Pow(2, retryAttempt)))
|
.WaitAndRetryAsync(config.MaxRetries, retryAttempt => TimeSpan.FromSeconds(Math.Pow(2, retryAttempt)))
|
||||||
);
|
);
|
||||||
|
|
||||||
|
private static IServiceCollection AddDryRunInterceptor(this IServiceCollection services)
|
||||||
|
{
|
||||||
|
services
|
||||||
|
.Where(s => s.ServiceType != typeof(IDryRunService) && typeof(IDryRunService).IsAssignableFrom(s.ServiceType))
|
||||||
|
.ToList()
|
||||||
|
.ForEach(service =>
|
||||||
|
{
|
||||||
|
services.Decorate(service.ServiceType, (target, svc) =>
|
||||||
|
{
|
||||||
|
ProxyGenerator proxyGenerator = new();
|
||||||
|
DryRunAsyncInterceptor interceptor = svc.GetRequiredService<DryRunAsyncInterceptor>();
|
||||||
|
|
||||||
|
object implementation = proxyGenerator.CreateClassProxyWithTarget(
|
||||||
|
service.ServiceType,
|
||||||
|
target,
|
||||||
|
interceptor
|
||||||
|
);
|
||||||
|
|
||||||
|
((IInterceptedService)target).Proxy = implementation;
|
||||||
|
|
||||||
|
return implementation;
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
return services;
|
||||||
|
}
|
||||||
}
|
}
|
||||||
@@ -1,10 +1,12 @@
|
|||||||
using Common.Configuration;
|
using Common.Configuration;
|
||||||
using Common.Configuration.ContentBlocker;
|
using Common.Configuration.ContentBlocker;
|
||||||
|
using Common.Configuration.DownloadCleaner;
|
||||||
using Common.Configuration.General;
|
using Common.Configuration.General;
|
||||||
using Common.Configuration.QueueCleaner;
|
using Common.Configuration.QueueCleaner;
|
||||||
using Common.Helpers;
|
using Common.Helpers;
|
||||||
using Executable.Jobs;
|
using Executable.Jobs;
|
||||||
using Infrastructure.Verticals.ContentBlocker;
|
using Infrastructure.Verticals.ContentBlocker;
|
||||||
|
using Infrastructure.Verticals.DownloadCleaner;
|
||||||
using Infrastructure.Verticals.Jobs;
|
using Infrastructure.Verticals.Jobs;
|
||||||
using Infrastructure.Verticals.QueueCleaner;
|
using Infrastructure.Verticals.QueueCleaner;
|
||||||
using Quartz;
|
using Quartz;
|
||||||
@@ -59,6 +61,12 @@ public static class QuartzDI
|
|||||||
{
|
{
|
||||||
q.AddJob<QueueCleaner>(queueCleanerConfig, triggersConfig.QueueCleaner);
|
q.AddJob<QueueCleaner>(queueCleanerConfig, triggersConfig.QueueCleaner);
|
||||||
}
|
}
|
||||||
|
|
||||||
|
DownloadCleanerConfig? downloadCleanerConfig = configuration
|
||||||
|
.GetRequiredSection(DownloadCleanerConfig.SectionName)
|
||||||
|
.Get<DownloadCleanerConfig>();
|
||||||
|
|
||||||
|
q.AddJob<DownloadCleaner>(downloadCleanerConfig, triggersConfig.DownloadCleaner);
|
||||||
}
|
}
|
||||||
|
|
||||||
private static void AddJob<T>(
|
private static void AddJob<T>(
|
||||||
@@ -109,7 +117,7 @@ public static class QuartzDI
|
|||||||
|
|
||||||
if (triggerValue > Constants.TriggerMaxLimit)
|
if (triggerValue > Constants.TriggerMaxLimit)
|
||||||
{
|
{
|
||||||
throw new Exception($"{trigger} should have a fire time of maximum 1 hour");
|
throw new Exception($"{trigger} should have a fire time of maximum {Constants.TriggerMaxLimit.TotalHours} hours");
|
||||||
}
|
}
|
||||||
|
|
||||||
if (triggerValue > StaticConfiguration.TriggerValue)
|
if (triggerValue > StaticConfiguration.TriggerValue)
|
||||||
|
|||||||
@@ -1,5 +1,7 @@
|
|||||||
using Infrastructure.Verticals.Arr;
|
using Infrastructure.Interceptors;
|
||||||
|
using Infrastructure.Verticals.Arr;
|
||||||
using Infrastructure.Verticals.ContentBlocker;
|
using Infrastructure.Verticals.ContentBlocker;
|
||||||
|
using Infrastructure.Verticals.DownloadCleaner;
|
||||||
using Infrastructure.Verticals.DownloadClient;
|
using Infrastructure.Verticals.DownloadClient;
|
||||||
using Infrastructure.Verticals.DownloadClient.Deluge;
|
using Infrastructure.Verticals.DownloadClient.Deluge;
|
||||||
using Infrastructure.Verticals.DownloadClient.QBittorrent;
|
using Infrastructure.Verticals.DownloadClient.QBittorrent;
|
||||||
@@ -13,12 +15,14 @@ public static class ServicesDI
|
|||||||
{
|
{
|
||||||
public static IServiceCollection AddServices(this IServiceCollection services) =>
|
public static IServiceCollection AddServices(this IServiceCollection services) =>
|
||||||
services
|
services
|
||||||
|
.AddTransient<DryRunAsyncInterceptor>()
|
||||||
.AddTransient<SonarrClient>()
|
.AddTransient<SonarrClient>()
|
||||||
.AddTransient<RadarrClient>()
|
.AddTransient<RadarrClient>()
|
||||||
.AddTransient<LidarrClient>()
|
.AddTransient<LidarrClient>()
|
||||||
.AddTransient<QueueCleaner>()
|
.AddTransient<QueueCleaner>()
|
||||||
.AddTransient<ContentBlocker>()
|
.AddTransient<ContentBlocker>()
|
||||||
.AddTransient<FilenameEvaluator>()
|
.AddTransient<DownloadCleaner>()
|
||||||
|
.AddTransient<IFilenameEvaluator, FilenameEvaluator>()
|
||||||
.AddTransient<DummyDownloadService>()
|
.AddTransient<DummyDownloadService>()
|
||||||
.AddTransient<QBitService>()
|
.AddTransient<QBitService>()
|
||||||
.AddTransient<DelugeService>()
|
.AddTransient<DelugeService>()
|
||||||
@@ -26,5 +30,5 @@ public static class ServicesDI
|
|||||||
.AddTransient<ArrQueueIterator>()
|
.AddTransient<ArrQueueIterator>()
|
||||||
.AddTransient<DownloadServiceFactory>()
|
.AddTransient<DownloadServiceFactory>()
|
||||||
.AddSingleton<BlocklistProvider>()
|
.AddSingleton<BlocklistProvider>()
|
||||||
.AddSingleton<Striker>();
|
.AddSingleton<IStriker, Striker>();
|
||||||
}
|
}
|
||||||
@@ -10,9 +10,9 @@
|
|||||||
</PropertyGroup>
|
</PropertyGroup>
|
||||||
|
|
||||||
<ItemGroup>
|
<ItemGroup>
|
||||||
<PackageReference Include="Microsoft.Extensions.Hosting" Version="9.0.0" />
|
<PackageReference Include="Microsoft.Extensions.Hosting" Version="9.0.2" />
|
||||||
<PackageReference Include="Microsoft.Extensions.Http" Version="9.0.0" />
|
<PackageReference Include="Microsoft.Extensions.Http" Version="9.0.2" />
|
||||||
<PackageReference Include="Microsoft.Extensions.Http.Polly" Version="9.0.0" />
|
<PackageReference Include="Microsoft.Extensions.Http.Polly" Version="9.0.2" />
|
||||||
<PackageReference Include="Quartz" Version="3.13.1" />
|
<PackageReference Include="Quartz" Version="3.13.1" />
|
||||||
<PackageReference Include="Quartz.Extensions.DependencyInjection" Version="3.13.1" />
|
<PackageReference Include="Quartz.Extensions.DependencyInjection" Version="3.13.1" />
|
||||||
<PackageReference Include="Quartz.Extensions.Hosting" Version="3.13.1" />
|
<PackageReference Include="Quartz.Extensions.Hosting" Version="3.13.1" />
|
||||||
|
|||||||
@@ -6,7 +6,7 @@ namespace Executable.Jobs;
|
|||||||
|
|
||||||
[DisallowConcurrentExecution]
|
[DisallowConcurrentExecution]
|
||||||
public sealed class GenericJob<T> : IJob
|
public sealed class GenericJob<T> : IJob
|
||||||
where T : GenericHandler
|
where T : IHandler
|
||||||
{
|
{
|
||||||
private readonly ILogger<GenericJob<T>> _logger;
|
private readonly ILogger<GenericJob<T>> _logger;
|
||||||
private readonly T _handler;
|
private readonly T _handler;
|
||||||
|
|||||||
@@ -1,4 +1,5 @@
|
|||||||
{
|
{
|
||||||
|
"DRY_RUN": true,
|
||||||
"HTTP_MAX_RETRIES": 0,
|
"HTTP_MAX_RETRIES": 0,
|
||||||
"HTTP_TIMEOUT": 10,
|
"HTTP_TIMEOUT": 10,
|
||||||
"Logging": {
|
"Logging": {
|
||||||
@@ -11,7 +12,8 @@
|
|||||||
},
|
},
|
||||||
"Triggers": {
|
"Triggers": {
|
||||||
"QueueCleaner": "0/10 * * * * ?",
|
"QueueCleaner": "0/10 * * * * ?",
|
||||||
"ContentBlocker": "0/10 * * * * ?"
|
"ContentBlocker": "0/10 * * * * ?",
|
||||||
|
"DownloadCleaner": "0/10 * * * * ?"
|
||||||
},
|
},
|
||||||
"ContentBlocker": {
|
"ContentBlocker": {
|
||||||
"Enabled": true,
|
"Enabled": true,
|
||||||
@@ -32,6 +34,18 @@
|
|||||||
"STALLED_IGNORE_PRIVATE": true,
|
"STALLED_IGNORE_PRIVATE": true,
|
||||||
"STALLED_DELETE_PRIVATE": false
|
"STALLED_DELETE_PRIVATE": false
|
||||||
},
|
},
|
||||||
|
"DownloadCleaner": {
|
||||||
|
"Enabled": false,
|
||||||
|
"DELETE_PRIVATE": false,
|
||||||
|
"CATEGORIES": [
|
||||||
|
{
|
||||||
|
"Name": "tv-sonarr",
|
||||||
|
"MAX_RATIO": -1,
|
||||||
|
"MIN_SEED_TIME": 0,
|
||||||
|
"MAX_SEED_TIME": -1
|
||||||
|
}
|
||||||
|
]
|
||||||
|
},
|
||||||
"DOWNLOAD_CLIENT": "qbittorrent",
|
"DOWNLOAD_CLIENT": "qbittorrent",
|
||||||
"qBittorrent": {
|
"qBittorrent": {
|
||||||
"Url": "http://localhost:8080",
|
"Url": "http://localhost:8080",
|
||||||
@@ -90,7 +104,8 @@
|
|||||||
"Notifiarr": {
|
"Notifiarr": {
|
||||||
"ON_IMPORT_FAILED_STRIKE": true,
|
"ON_IMPORT_FAILED_STRIKE": true,
|
||||||
"ON_STALLED_STRIKE": true,
|
"ON_STALLED_STRIKE": true,
|
||||||
"ON_QUEUE_ITEM_DELETE": true,
|
"ON_QUEUE_ITEM_DELETED": true,
|
||||||
|
"ON_DOWNLOAD_CLEANED": true,
|
||||||
"API_KEY": "",
|
"API_KEY": "",
|
||||||
"CHANNEL_ID": ""
|
"CHANNEL_ID": ""
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -1,4 +1,5 @@
|
|||||||
{
|
{
|
||||||
|
"DRY_RUN": false,
|
||||||
"HTTP_MAX_RETRIES": 0,
|
"HTTP_MAX_RETRIES": 0,
|
||||||
"HTTP_TIMEOUT": 100,
|
"HTTP_TIMEOUT": 100,
|
||||||
"Logging": {
|
"Logging": {
|
||||||
@@ -11,7 +12,8 @@
|
|||||||
},
|
},
|
||||||
"Triggers": {
|
"Triggers": {
|
||||||
"QueueCleaner": "0 0/5 * * * ?",
|
"QueueCleaner": "0 0/5 * * * ?",
|
||||||
"ContentBlocker": "0 0/5 * * * ?"
|
"ContentBlocker": "0 0/5 * * * ?",
|
||||||
|
"DownloadCleaner": "0 0 * * * ?"
|
||||||
},
|
},
|
||||||
"ContentBlocker": {
|
"ContentBlocker": {
|
||||||
"Enabled": false,
|
"Enabled": false,
|
||||||
@@ -29,6 +31,11 @@
|
|||||||
"STALLED_IGNORE_PRIVATE": false,
|
"STALLED_IGNORE_PRIVATE": false,
|
||||||
"STALLED_DELETE_PRIVATE": false
|
"STALLED_DELETE_PRIVATE": false
|
||||||
},
|
},
|
||||||
|
"DownloadCleaner": {
|
||||||
|
"Enabled": false,
|
||||||
|
"DELETE_PRIVATE": false,
|
||||||
|
"CATEGORIES": []
|
||||||
|
},
|
||||||
"DOWNLOAD_CLIENT": "none",
|
"DOWNLOAD_CLIENT": "none",
|
||||||
"qBittorrent": {
|
"qBittorrent": {
|
||||||
"Url": "http://localhost:8080",
|
"Url": "http://localhost:8080",
|
||||||
@@ -87,7 +94,8 @@
|
|||||||
"Notifiarr": {
|
"Notifiarr": {
|
||||||
"ON_IMPORT_FAILED_STRIKE": false,
|
"ON_IMPORT_FAILED_STRIKE": false,
|
||||||
"ON_STALLED_STRIKE": false,
|
"ON_STALLED_STRIKE": false,
|
||||||
"ON_QUEUE_ITEM_DELETE": false,
|
"ON_QUEUE_ITEM_DELETED": false,
|
||||||
|
"ON_DOWNLOAD_CLEANED": false,
|
||||||
"API_KEY": "",
|
"API_KEY": "",
|
||||||
"CHANNEL_ID": ""
|
"CHANNEL_ID": ""
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -0,0 +1,33 @@
|
|||||||
|
<Project Sdk="Microsoft.NET.Sdk">
|
||||||
|
|
||||||
|
<PropertyGroup>
|
||||||
|
<TargetFramework>net9.0</TargetFramework>
|
||||||
|
<ImplicitUsings>enable</ImplicitUsings>
|
||||||
|
<Nullable>enable</Nullable>
|
||||||
|
<IsPackable>false</IsPackable>
|
||||||
|
</PropertyGroup>
|
||||||
|
|
||||||
|
<ItemGroup>
|
||||||
|
<PackageReference Include="coverlet.collector" Version="6.0.4">
|
||||||
|
<PrivateAssets>all</PrivateAssets>
|
||||||
|
<IncludeAssets>runtime; build; native; contentfiles; analyzers; buildtransitive</IncludeAssets>
|
||||||
|
</PackageReference>
|
||||||
|
<PackageReference Include="Microsoft.NET.Test.Sdk" Version="17.13.0" />
|
||||||
|
<PackageReference Include="NSubstitute" Version="5.3.0" />
|
||||||
|
<PackageReference Include="Shouldly" Version="4.3.0" />
|
||||||
|
<PackageReference Include="xunit" Version="2.9.3" />
|
||||||
|
<PackageReference Include="xunit.runner.visualstudio" Version="3.0.2">
|
||||||
|
<PrivateAssets>all</PrivateAssets>
|
||||||
|
<IncludeAssets>runtime; build; native; contentfiles; analyzers; buildtransitive</IncludeAssets>
|
||||||
|
</PackageReference>
|
||||||
|
</ItemGroup>
|
||||||
|
|
||||||
|
<ItemGroup>
|
||||||
|
<Using Include="Xunit"/>
|
||||||
|
</ItemGroup>
|
||||||
|
|
||||||
|
<ItemGroup>
|
||||||
|
<ProjectReference Include="..\Infrastructure\Infrastructure.csproj" />
|
||||||
|
</ItemGroup>
|
||||||
|
|
||||||
|
</Project>
|
||||||
@@ -0,0 +1,20 @@
|
|||||||
|
using Infrastructure.Verticals.ContentBlocker;
|
||||||
|
using Microsoft.Extensions.Logging;
|
||||||
|
using NSubstitute;
|
||||||
|
|
||||||
|
namespace Infrastructure.Tests.Verticals.ContentBlocker;
|
||||||
|
|
||||||
|
public class FilenameEvaluatorFixture
|
||||||
|
{
|
||||||
|
public ILogger<FilenameEvaluator> Logger { get; }
|
||||||
|
|
||||||
|
public FilenameEvaluatorFixture()
|
||||||
|
{
|
||||||
|
Logger = Substitute.For<ILogger<FilenameEvaluator>>();
|
||||||
|
}
|
||||||
|
|
||||||
|
public FilenameEvaluator CreateSut()
|
||||||
|
{
|
||||||
|
return new FilenameEvaluator(Logger);
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,219 @@
|
|||||||
|
using System.Collections.Concurrent;
|
||||||
|
using System.Text.RegularExpressions;
|
||||||
|
using Common.Configuration.ContentBlocker;
|
||||||
|
using Shouldly;
|
||||||
|
|
||||||
|
namespace Infrastructure.Tests.Verticals.ContentBlocker;
|
||||||
|
|
||||||
|
public class FilenameEvaluatorTests : IClassFixture<FilenameEvaluatorFixture>
|
||||||
|
{
|
||||||
|
private readonly FilenameEvaluatorFixture _fixture;
|
||||||
|
|
||||||
|
public FilenameEvaluatorTests(FilenameEvaluatorFixture fixture)
|
||||||
|
{
|
||||||
|
_fixture = fixture;
|
||||||
|
}
|
||||||
|
|
||||||
|
public class PatternTests : FilenameEvaluatorTests
|
||||||
|
{
|
||||||
|
public PatternTests(FilenameEvaluatorFixture fixture) : base(fixture) { }
|
||||||
|
|
||||||
|
[Fact]
|
||||||
|
public void WhenNoPatterns_ShouldReturnTrue()
|
||||||
|
{
|
||||||
|
// Arrange
|
||||||
|
var sut = _fixture.CreateSut();
|
||||||
|
var patterns = new ConcurrentBag<string>();
|
||||||
|
var regexes = new ConcurrentBag<Regex>();
|
||||||
|
|
||||||
|
// Act
|
||||||
|
var result = sut.IsValid("test.txt", BlocklistType.Blacklist, patterns, regexes);
|
||||||
|
|
||||||
|
// Assert
|
||||||
|
result.ShouldBeTrue();
|
||||||
|
}
|
||||||
|
|
||||||
|
[Theory]
|
||||||
|
[InlineData("test.txt", "test.txt", true)] // Exact match
|
||||||
|
[InlineData("test.txt", "*.txt", true)] // End wildcard
|
||||||
|
[InlineData("test.txt", "test.*", true)] // Start wildcard
|
||||||
|
[InlineData("test.txt", "*test*", true)] // Both wildcards
|
||||||
|
[InlineData("test.txt", "other.txt", false)] // No match
|
||||||
|
public void Blacklist_ShouldMatchPatterns(string filename, string pattern, bool shouldBeBlocked)
|
||||||
|
{
|
||||||
|
// Arrange
|
||||||
|
var sut = _fixture.CreateSut();
|
||||||
|
var patterns = new ConcurrentBag<string> { pattern };
|
||||||
|
var regexes = new ConcurrentBag<Regex>();
|
||||||
|
|
||||||
|
// Act
|
||||||
|
var result = sut.IsValid(filename, BlocklistType.Blacklist, patterns, regexes);
|
||||||
|
|
||||||
|
// Assert
|
||||||
|
result.ShouldBe(!shouldBeBlocked);
|
||||||
|
}
|
||||||
|
|
||||||
|
[Theory]
|
||||||
|
[InlineData("test.txt", "test.txt", true)] // Exact match
|
||||||
|
[InlineData("test.txt", "*.txt", true)] // End wildcard
|
||||||
|
[InlineData("test.txt", "test.*", true)] // Start wildcard
|
||||||
|
[InlineData("test.txt", "*test*", true)] // Both wildcards
|
||||||
|
[InlineData("test.txt", "other.txt", false)] // No match
|
||||||
|
public void Whitelist_ShouldMatchPatterns(string filename, string pattern, bool shouldBeAllowed)
|
||||||
|
{
|
||||||
|
// Arrange
|
||||||
|
var sut = _fixture.CreateSut();
|
||||||
|
var patterns = new ConcurrentBag<string> { pattern };
|
||||||
|
var regexes = new ConcurrentBag<Regex>();
|
||||||
|
|
||||||
|
// Act
|
||||||
|
var result = sut.IsValid(filename, BlocklistType.Whitelist, patterns, regexes);
|
||||||
|
|
||||||
|
// Assert
|
||||||
|
result.ShouldBe(shouldBeAllowed);
|
||||||
|
}
|
||||||
|
|
||||||
|
[Theory]
|
||||||
|
[InlineData("TEST.TXT", "test.txt")]
|
||||||
|
[InlineData("test.txt", "TEST.TXT")]
|
||||||
|
public void ShouldBeCaseInsensitive(string filename, string pattern)
|
||||||
|
{
|
||||||
|
// Arrange
|
||||||
|
var sut = _fixture.CreateSut();
|
||||||
|
var patterns = new ConcurrentBag<string> { pattern };
|
||||||
|
var regexes = new ConcurrentBag<Regex>();
|
||||||
|
|
||||||
|
// Act
|
||||||
|
var result = sut.IsValid(filename, BlocklistType.Blacklist, patterns, regexes);
|
||||||
|
|
||||||
|
// Assert
|
||||||
|
result.ShouldBeFalse();
|
||||||
|
}
|
||||||
|
|
||||||
|
[Fact]
|
||||||
|
public void MultiplePatterns_ShouldMatchAny()
|
||||||
|
{
|
||||||
|
// Arrange
|
||||||
|
var sut = _fixture.CreateSut();
|
||||||
|
var patterns = new ConcurrentBag<string>
|
||||||
|
{
|
||||||
|
"other.txt",
|
||||||
|
"*.pdf",
|
||||||
|
"test.*"
|
||||||
|
};
|
||||||
|
var regexes = new ConcurrentBag<Regex>();
|
||||||
|
|
||||||
|
// Act
|
||||||
|
var result = sut.IsValid("test.txt", BlocklistType.Blacklist, patterns, regexes);
|
||||||
|
|
||||||
|
// Assert
|
||||||
|
result.ShouldBeFalse();
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
public class RegexTests : FilenameEvaluatorTests
|
||||||
|
{
|
||||||
|
public RegexTests(FilenameEvaluatorFixture fixture) : base(fixture) { }
|
||||||
|
|
||||||
|
[Fact]
|
||||||
|
public void WhenNoRegexes_ShouldReturnTrue()
|
||||||
|
{
|
||||||
|
// Arrange
|
||||||
|
var sut = _fixture.CreateSut();
|
||||||
|
var patterns = new ConcurrentBag<string>();
|
||||||
|
var regexes = new ConcurrentBag<Regex>();
|
||||||
|
|
||||||
|
// Act
|
||||||
|
var result = sut.IsValid("test.txt", BlocklistType.Blacklist, patterns, regexes);
|
||||||
|
|
||||||
|
// Assert
|
||||||
|
result.ShouldBeTrue();
|
||||||
|
}
|
||||||
|
|
||||||
|
[Theory]
|
||||||
|
[InlineData(@"test\d+\.txt", "test123.txt", true)]
|
||||||
|
[InlineData(@"test\d+\.txt", "test.txt", false)]
|
||||||
|
public void Blacklist_ShouldMatchRegexes(string pattern, string filename, bool shouldBeBlocked)
|
||||||
|
{
|
||||||
|
// Arrange
|
||||||
|
var sut = _fixture.CreateSut();
|
||||||
|
var patterns = new ConcurrentBag<string>();
|
||||||
|
var regexes = new ConcurrentBag<Regex> { new Regex(pattern, RegexOptions.IgnoreCase) };
|
||||||
|
|
||||||
|
// Act
|
||||||
|
var result = sut.IsValid(filename, BlocklistType.Blacklist, patterns, regexes);
|
||||||
|
|
||||||
|
// Assert
|
||||||
|
result.ShouldBe(!shouldBeBlocked);
|
||||||
|
}
|
||||||
|
|
||||||
|
[Theory]
|
||||||
|
[InlineData(@"test\d+\.txt", "test123.txt", true)]
|
||||||
|
[InlineData(@"test\d+\.txt", "test.txt", false)]
|
||||||
|
public void Whitelist_ShouldMatchRegexes(string pattern, string filename, bool shouldBeAllowed)
|
||||||
|
{
|
||||||
|
// Arrange
|
||||||
|
var sut = _fixture.CreateSut();
|
||||||
|
var patterns = new ConcurrentBag<string>();
|
||||||
|
var regexes = new ConcurrentBag<Regex> { new Regex(pattern, RegexOptions.IgnoreCase) };
|
||||||
|
|
||||||
|
// Act
|
||||||
|
var result = sut.IsValid(filename, BlocklistType.Whitelist, patterns, regexes);
|
||||||
|
|
||||||
|
// Assert
|
||||||
|
result.ShouldBe(shouldBeAllowed);
|
||||||
|
}
|
||||||
|
|
||||||
|
[Theory]
|
||||||
|
[InlineData(@"TEST\d+\.TXT", "test123.txt")]
|
||||||
|
[InlineData(@"test\d+\.txt", "TEST123.TXT")]
|
||||||
|
public void ShouldBeCaseInsensitive(string pattern, string filename)
|
||||||
|
{
|
||||||
|
// Arrange
|
||||||
|
var sut = _fixture.CreateSut();
|
||||||
|
var patterns = new ConcurrentBag<string>();
|
||||||
|
var regexes = new ConcurrentBag<Regex> { new Regex(pattern, RegexOptions.IgnoreCase) };
|
||||||
|
|
||||||
|
// Act
|
||||||
|
var result = sut.IsValid(filename, BlocklistType.Blacklist, patterns, regexes);
|
||||||
|
|
||||||
|
// Assert
|
||||||
|
result.ShouldBeFalse();
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
public class CombinedTests : FilenameEvaluatorTests
|
||||||
|
{
|
||||||
|
public CombinedTests(FilenameEvaluatorFixture fixture) : base(fixture) { }
|
||||||
|
|
||||||
|
[Fact]
|
||||||
|
public void WhenBothPatternsAndRegexes_ShouldMatchBoth()
|
||||||
|
{
|
||||||
|
// Arrange
|
||||||
|
var sut = _fixture.CreateSut();
|
||||||
|
var patterns = new ConcurrentBag<string> { "*.txt" };
|
||||||
|
var regexes = new ConcurrentBag<Regex> { new Regex(@"test\d+", RegexOptions.IgnoreCase) };
|
||||||
|
|
||||||
|
// Act
|
||||||
|
var result = sut.IsValid("test123.txt", BlocklistType.Blacklist, patterns, regexes);
|
||||||
|
|
||||||
|
// Assert
|
||||||
|
result.ShouldBeFalse();
|
||||||
|
}
|
||||||
|
|
||||||
|
[Fact]
|
||||||
|
public void WhenPatternMatchesButRegexDoesNot_ShouldReturnFalse()
|
||||||
|
{
|
||||||
|
// Arrange
|
||||||
|
var sut = _fixture.CreateSut();
|
||||||
|
var patterns = new ConcurrentBag<string> { "*.txt" };
|
||||||
|
var regexes = new ConcurrentBag<Regex> { new Regex(@"test\d+", RegexOptions.IgnoreCase) };
|
||||||
|
|
||||||
|
// Act
|
||||||
|
var result = sut.IsValid("other.txt", BlocklistType.Whitelist, patterns, regexes);
|
||||||
|
|
||||||
|
// Assert
|
||||||
|
result.ShouldBeFalse();
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,74 @@
|
|||||||
|
using Common.Configuration.ContentBlocker;
|
||||||
|
using Common.Configuration.DownloadCleaner;
|
||||||
|
using Common.Configuration.QueueCleaner;
|
||||||
|
using Infrastructure.Verticals.ContentBlocker;
|
||||||
|
using Infrastructure.Verticals.DownloadClient;
|
||||||
|
using Infrastructure.Verticals.ItemStriker;
|
||||||
|
using Infrastructure.Verticals.Notifications;
|
||||||
|
using Microsoft.Extensions.Caching.Memory;
|
||||||
|
using Microsoft.Extensions.Logging;
|
||||||
|
using Microsoft.Extensions.Options;
|
||||||
|
using NSubstitute;
|
||||||
|
|
||||||
|
namespace Infrastructure.Tests.Verticals.DownloadClient;
|
||||||
|
|
||||||
|
public class DownloadServiceFixture : IDisposable
|
||||||
|
{
|
||||||
|
public ILogger<DownloadService> Logger { get; set; }
|
||||||
|
public IMemoryCache Cache { get; set; }
|
||||||
|
public IStriker Striker { get; set; }
|
||||||
|
|
||||||
|
public DownloadServiceFixture()
|
||||||
|
{
|
||||||
|
Logger = Substitute.For<ILogger<DownloadService>>();
|
||||||
|
Cache = Substitute.For<IMemoryCache>();
|
||||||
|
Striker = Substitute.For<IStriker>();
|
||||||
|
}
|
||||||
|
|
||||||
|
public TestDownloadService CreateSut(
|
||||||
|
QueueCleanerConfig? queueCleanerConfig = null,
|
||||||
|
ContentBlockerConfig? contentBlockerConfig = null
|
||||||
|
)
|
||||||
|
{
|
||||||
|
queueCleanerConfig ??= new QueueCleanerConfig
|
||||||
|
{
|
||||||
|
Enabled = true,
|
||||||
|
RunSequentially = true,
|
||||||
|
StalledResetStrikesOnProgress = true,
|
||||||
|
StalledMaxStrikes = 3
|
||||||
|
};
|
||||||
|
|
||||||
|
var queueCleanerOptions = Substitute.For<IOptions<QueueCleanerConfig>>();
|
||||||
|
queueCleanerOptions.Value.Returns(queueCleanerConfig);
|
||||||
|
|
||||||
|
contentBlockerConfig ??= new ContentBlockerConfig
|
||||||
|
{
|
||||||
|
Enabled = true
|
||||||
|
};
|
||||||
|
|
||||||
|
var contentBlockerOptions = Substitute.For<IOptions<ContentBlockerConfig>>();
|
||||||
|
contentBlockerOptions.Value.Returns(contentBlockerConfig);
|
||||||
|
|
||||||
|
var downloadCleanerOptions = Substitute.For<IOptions<DownloadCleanerConfig>>();
|
||||||
|
downloadCleanerOptions.Value.Returns(new DownloadCleanerConfig());
|
||||||
|
|
||||||
|
var filenameEvaluator = Substitute.For<IFilenameEvaluator>();
|
||||||
|
var notifier = Substitute.For<NotificationPublisher>();
|
||||||
|
|
||||||
|
return new TestDownloadService(
|
||||||
|
Logger,
|
||||||
|
queueCleanerOptions,
|
||||||
|
contentBlockerOptions,
|
||||||
|
downloadCleanerOptions,
|
||||||
|
Cache,
|
||||||
|
filenameEvaluator,
|
||||||
|
Striker,
|
||||||
|
notifier
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
public void Dispose()
|
||||||
|
{
|
||||||
|
// Cleanup if needed
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,235 @@
|
|||||||
|
using Common.Configuration.DownloadCleaner;
|
||||||
|
using Domain.Enums;
|
||||||
|
using Domain.Models.Cache;
|
||||||
|
using Infrastructure.Helpers;
|
||||||
|
using Infrastructure.Verticals.Context;
|
||||||
|
using Infrastructure.Verticals.DownloadClient;
|
||||||
|
using NSubstitute;
|
||||||
|
using NSubstitute.ClearExtensions;
|
||||||
|
using Shouldly;
|
||||||
|
|
||||||
|
namespace Infrastructure.Tests.Verticals.DownloadClient;
|
||||||
|
|
||||||
|
public class DownloadServiceTests : IClassFixture<DownloadServiceFixture>
|
||||||
|
{
|
||||||
|
private readonly DownloadServiceFixture _fixture;
|
||||||
|
|
||||||
|
public DownloadServiceTests(DownloadServiceFixture fixture)
|
||||||
|
{
|
||||||
|
_fixture = fixture;
|
||||||
|
_fixture.Cache.ClearSubstitute();
|
||||||
|
_fixture.Striker.ClearSubstitute();
|
||||||
|
}
|
||||||
|
|
||||||
|
public class ResetStrikesOnProgressTests : DownloadServiceTests
|
||||||
|
{
|
||||||
|
public ResetStrikesOnProgressTests(DownloadServiceFixture fixture) : base(fixture)
|
||||||
|
{
|
||||||
|
}
|
||||||
|
|
||||||
|
[Fact]
|
||||||
|
public void WhenStalledStrikeDisabled_ShouldNotResetStrikes()
|
||||||
|
{
|
||||||
|
// Arrange
|
||||||
|
TestDownloadService sut = _fixture.CreateSut(queueCleanerConfig: new()
|
||||||
|
{
|
||||||
|
Enabled = true,
|
||||||
|
RunSequentially = true,
|
||||||
|
StalledResetStrikesOnProgress = false,
|
||||||
|
});
|
||||||
|
|
||||||
|
// Act
|
||||||
|
sut.ResetStrikesOnProgress("test-hash", 100);
|
||||||
|
|
||||||
|
// Assert
|
||||||
|
_fixture.Cache.ReceivedCalls().ShouldBeEmpty();
|
||||||
|
}
|
||||||
|
|
||||||
|
[Fact]
|
||||||
|
public void WhenProgressMade_ShouldResetStrikes()
|
||||||
|
{
|
||||||
|
// Arrange
|
||||||
|
const string hash = "test-hash";
|
||||||
|
CacheItem cacheItem = new CacheItem { Downloaded = 100 };
|
||||||
|
|
||||||
|
_fixture.Cache.TryGetValue(Arg.Any<object>(), out Arg.Any<object?>())
|
||||||
|
.Returns(x =>
|
||||||
|
{
|
||||||
|
x[1] = cacheItem;
|
||||||
|
return true;
|
||||||
|
});
|
||||||
|
|
||||||
|
TestDownloadService sut = _fixture.CreateSut();
|
||||||
|
|
||||||
|
// Act
|
||||||
|
sut.ResetStrikesOnProgress(hash, 200);
|
||||||
|
|
||||||
|
// Assert
|
||||||
|
_fixture.Cache.Received(1).Remove(CacheKeys.Strike(StrikeType.Stalled, hash));
|
||||||
|
}
|
||||||
|
|
||||||
|
[Fact]
|
||||||
|
public void WhenNoProgress_ShouldNotResetStrikes()
|
||||||
|
{
|
||||||
|
// Arrange
|
||||||
|
const string hash = "test-hash";
|
||||||
|
CacheItem cacheItem = new CacheItem { Downloaded = 200 };
|
||||||
|
|
||||||
|
_fixture.Cache
|
||||||
|
.TryGetValue(Arg.Any<object>(), out Arg.Any<object?>())
|
||||||
|
.Returns(x =>
|
||||||
|
{
|
||||||
|
x[1] = cacheItem;
|
||||||
|
return true;
|
||||||
|
});
|
||||||
|
|
||||||
|
TestDownloadService sut = _fixture.CreateSut();
|
||||||
|
|
||||||
|
// Act
|
||||||
|
sut.ResetStrikesOnProgress(hash, 100);
|
||||||
|
|
||||||
|
// Assert
|
||||||
|
_fixture.Cache.DidNotReceive().Remove(Arg.Any<object>());
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
public class StrikeAndCheckLimitTests : DownloadServiceTests
|
||||||
|
{
|
||||||
|
public StrikeAndCheckLimitTests(DownloadServiceFixture fixture) : base(fixture)
|
||||||
|
{
|
||||||
|
}
|
||||||
|
|
||||||
|
[Fact]
|
||||||
|
public async Task ShouldDelegateCallToStriker()
|
||||||
|
{
|
||||||
|
// Arrange
|
||||||
|
const string hash = "test-hash";
|
||||||
|
const string itemName = "test-item";
|
||||||
|
_fixture.Striker.StrikeAndCheckLimit(hash, itemName, 3, StrikeType.Stalled)
|
||||||
|
.Returns(true);
|
||||||
|
|
||||||
|
TestDownloadService sut = _fixture.CreateSut();
|
||||||
|
|
||||||
|
// Act
|
||||||
|
bool result = await sut.StrikeAndCheckLimit(hash, itemName);
|
||||||
|
|
||||||
|
// Assert
|
||||||
|
result.ShouldBeTrue();
|
||||||
|
await _fixture.Striker
|
||||||
|
.Received(1)
|
||||||
|
.StrikeAndCheckLimit(hash, itemName, 3, StrikeType.Stalled);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
public class ShouldCleanDownloadTests : DownloadServiceTests
|
||||||
|
{
|
||||||
|
public ShouldCleanDownloadTests(DownloadServiceFixture fixture) : base(fixture)
|
||||||
|
{
|
||||||
|
ContextProvider.Set("downloadName", "test-download");
|
||||||
|
}
|
||||||
|
|
||||||
|
[Fact]
|
||||||
|
public void WhenRatioAndMinSeedTimeReached_ShouldReturnTrue()
|
||||||
|
{
|
||||||
|
// Arrange
|
||||||
|
Category category = new()
|
||||||
|
{
|
||||||
|
Name = "test",
|
||||||
|
MaxRatio = 1.0,
|
||||||
|
MinSeedTime = 1,
|
||||||
|
MaxSeedTime = -1
|
||||||
|
};
|
||||||
|
const double ratio = 1.5;
|
||||||
|
TimeSpan seedingTime = TimeSpan.FromHours(2);
|
||||||
|
|
||||||
|
TestDownloadService sut = _fixture.CreateSut();
|
||||||
|
|
||||||
|
// Act
|
||||||
|
var result = sut.ShouldCleanDownload(ratio, seedingTime, category);
|
||||||
|
|
||||||
|
// Assert
|
||||||
|
result.ShouldSatisfyAllConditions(
|
||||||
|
() => result.ShouldClean.ShouldBeTrue(),
|
||||||
|
() => result.Reason.ShouldBe(CleanReason.MaxRatioReached)
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
[Fact]
|
||||||
|
public void WhenRatioReachedAndMinSeedTimeNotReached_ShouldReturnFalse()
|
||||||
|
{
|
||||||
|
// Arrange
|
||||||
|
Category category = new()
|
||||||
|
{
|
||||||
|
Name = "test",
|
||||||
|
MaxRatio = 1.0,
|
||||||
|
MinSeedTime = 3,
|
||||||
|
MaxSeedTime = -1
|
||||||
|
};
|
||||||
|
const double ratio = 1.5;
|
||||||
|
TimeSpan seedingTime = TimeSpan.FromHours(2);
|
||||||
|
|
||||||
|
TestDownloadService sut = _fixture.CreateSut();
|
||||||
|
|
||||||
|
// Act
|
||||||
|
var result = sut.ShouldCleanDownload(ratio, seedingTime, category);
|
||||||
|
|
||||||
|
// Assert
|
||||||
|
result.ShouldSatisfyAllConditions(
|
||||||
|
() => result.ShouldClean.ShouldBeFalse(),
|
||||||
|
() => result.Reason.ShouldBe(CleanReason.None)
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
[Fact]
|
||||||
|
public void WhenMaxSeedTimeReached_ShouldReturnTrue()
|
||||||
|
{
|
||||||
|
// Arrange
|
||||||
|
Category category = new()
|
||||||
|
{
|
||||||
|
Name = "test",
|
||||||
|
MaxRatio = -1,
|
||||||
|
MinSeedTime = 0,
|
||||||
|
MaxSeedTime = 1
|
||||||
|
};
|
||||||
|
const double ratio = 0.5;
|
||||||
|
TimeSpan seedingTime = TimeSpan.FromHours(2);
|
||||||
|
|
||||||
|
TestDownloadService sut = _fixture.CreateSut();
|
||||||
|
|
||||||
|
// Act
|
||||||
|
SeedingCheckResult result = sut.ShouldCleanDownload(ratio, seedingTime, category);
|
||||||
|
|
||||||
|
// Assert
|
||||||
|
result.ShouldSatisfyAllConditions(
|
||||||
|
() => result.ShouldClean.ShouldBeTrue(),
|
||||||
|
() => result.Reason.ShouldBe(CleanReason.MaxSeedTimeReached)
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
[Fact]
|
||||||
|
public void WhenNeitherConditionMet_ShouldReturnFalse()
|
||||||
|
{
|
||||||
|
// Arrange
|
||||||
|
Category category = new()
|
||||||
|
{
|
||||||
|
Name = "test",
|
||||||
|
MaxRatio = 2.0,
|
||||||
|
MinSeedTime = 0,
|
||||||
|
MaxSeedTime = 3
|
||||||
|
};
|
||||||
|
const double ratio = 1.0;
|
||||||
|
TimeSpan seedingTime = TimeSpan.FromHours(1);
|
||||||
|
|
||||||
|
TestDownloadService sut = _fixture.CreateSut();
|
||||||
|
|
||||||
|
// Act
|
||||||
|
var result = sut.ShouldCleanDownload(ratio, seedingTime, category);
|
||||||
|
|
||||||
|
// Assert
|
||||||
|
result.ShouldSatisfyAllConditions(
|
||||||
|
() => result.ShouldClean.ShouldBeFalse(),
|
||||||
|
() => result.Reason.ShouldBe(CleanReason.None)
|
||||||
|
);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,45 @@
|
|||||||
|
using System.Collections.Concurrent;
|
||||||
|
using System.Text.RegularExpressions;
|
||||||
|
using Common.Configuration.ContentBlocker;
|
||||||
|
using Common.Configuration.DownloadCleaner;
|
||||||
|
using Common.Configuration.QueueCleaner;
|
||||||
|
using Infrastructure.Verticals.ContentBlocker;
|
||||||
|
using Infrastructure.Verticals.DownloadClient;
|
||||||
|
using Infrastructure.Verticals.ItemStriker;
|
||||||
|
using Infrastructure.Verticals.Notifications;
|
||||||
|
using Microsoft.Extensions.Caching.Memory;
|
||||||
|
using Microsoft.Extensions.Logging;
|
||||||
|
using Microsoft.Extensions.Options;
|
||||||
|
|
||||||
|
namespace Infrastructure.Tests.Verticals.DownloadClient;
|
||||||
|
|
||||||
|
public class TestDownloadService : DownloadService
|
||||||
|
{
|
||||||
|
public TestDownloadService(
|
||||||
|
ILogger<DownloadService> logger,
|
||||||
|
IOptions<QueueCleanerConfig> queueCleanerConfig,
|
||||||
|
IOptions<ContentBlockerConfig> contentBlockerConfig,
|
||||||
|
IOptions<DownloadCleanerConfig> downloadCleanerConfig,
|
||||||
|
IMemoryCache cache,
|
||||||
|
IFilenameEvaluator filenameEvaluator,
|
||||||
|
IStriker striker,
|
||||||
|
NotificationPublisher notifier)
|
||||||
|
: base(logger, queueCleanerConfig, contentBlockerConfig, downloadCleanerConfig,
|
||||||
|
cache, filenameEvaluator, striker, notifier)
|
||||||
|
{
|
||||||
|
}
|
||||||
|
|
||||||
|
public override void Dispose() { }
|
||||||
|
public override Task LoginAsync() => Task.CompletedTask;
|
||||||
|
public override Task<StalledResult> ShouldRemoveFromArrQueueAsync(string hash) => Task.FromResult(new StalledResult());
|
||||||
|
public override Task<BlockFilesResult> BlockUnwantedFilesAsync(string hash, BlocklistType blocklistType,
|
||||||
|
ConcurrentBag<string> patterns, ConcurrentBag<Regex> regexes) => Task.FromResult(new BlockFilesResult());
|
||||||
|
public override Task DeleteDownload(string hash) => Task.CompletedTask;
|
||||||
|
public override Task<List<object>?> GetAllDownloadsToBeCleaned(List<Category> categories) => Task.FromResult<List<object>?>(null);
|
||||||
|
public override Task CleanDownloads(List<object> downloads, List<Category> categoriesToClean, HashSet<string> excludedHashes) => Task.CompletedTask;
|
||||||
|
|
||||||
|
// Expose protected methods for testing
|
||||||
|
public new void ResetStrikesOnProgress(string hash, long downloaded) => base.ResetStrikesOnProgress(hash, downloaded);
|
||||||
|
public new Task<bool> StrikeAndCheckLimit(string hash, string itemName) => base.StrikeAndCheckLimit(hash, itemName);
|
||||||
|
public new SeedingCheckResult ShouldCleanDownload(double ratio, TimeSpan seedingTime, Category category) => base.ShouldCleanDownload(ratio, seedingTime, category);
|
||||||
|
}
|
||||||
@@ -12,13 +12,15 @@
|
|||||||
</ItemGroup>
|
</ItemGroup>
|
||||||
|
|
||||||
<ItemGroup>
|
<ItemGroup>
|
||||||
|
<PackageReference Include="Castle.Core.AsyncInterceptor" Version="2.1.0" />
|
||||||
<PackageReference Include="FLM.QBittorrent" Version="1.0.0" />
|
<PackageReference Include="FLM.QBittorrent" Version="1.0.0" />
|
||||||
<PackageReference Include="FLM.Transmission" Version="1.0.2" />
|
<PackageReference Include="FLM.Transmission" Version="1.0.2" />
|
||||||
<PackageReference Include="Mapster" Version="7.4.0" />
|
<PackageReference Include="Mapster" Version="7.4.0" />
|
||||||
<PackageReference Include="MassTransit" Version="8.3.6" />
|
<PackageReference Include="MassTransit" Version="8.3.6" />
|
||||||
<PackageReference Include="Microsoft.Extensions.Caching.Memory" Version="9.0.0" />
|
<PackageReference Include="Microsoft.Extensions.Caching.Memory" Version="9.0.2" />
|
||||||
<PackageReference Include="Microsoft.Extensions.Http" Version="9.0.0" />
|
<PackageReference Include="Microsoft.Extensions.Http" Version="9.0.2" />
|
||||||
<PackageReference Include="Quartz" Version="3.13.1" />
|
<PackageReference Include="Quartz" Version="3.13.1" />
|
||||||
|
<PackageReference Include="Scrutor" Version="6.0.1" />
|
||||||
</ItemGroup>
|
</ItemGroup>
|
||||||
|
|
||||||
</Project>
|
</Project>
|
||||||
|
|||||||
@@ -0,0 +1,49 @@
|
|||||||
|
using System.Reflection;
|
||||||
|
using Castle.DynamicProxy;
|
||||||
|
using Common.Attributes;
|
||||||
|
using Common.Configuration.General;
|
||||||
|
using Microsoft.Extensions.Logging;
|
||||||
|
using Microsoft.Extensions.Options;
|
||||||
|
|
||||||
|
namespace Infrastructure.Interceptors;
|
||||||
|
|
||||||
|
public class DryRunAsyncInterceptor : AsyncInterceptorBase
|
||||||
|
{
|
||||||
|
private readonly ILogger<DryRunAsyncInterceptor> _logger;
|
||||||
|
private readonly DryRunConfig _config;
|
||||||
|
|
||||||
|
public DryRunAsyncInterceptor(ILogger<DryRunAsyncInterceptor> logger, IOptions<DryRunConfig> config)
|
||||||
|
{
|
||||||
|
_logger = logger;
|
||||||
|
_config = config.Value;
|
||||||
|
}
|
||||||
|
|
||||||
|
protected override async Task InterceptAsync(IInvocation invocation, IInvocationProceedInfo proceedInfo, Func<IInvocation, IInvocationProceedInfo, Task> proceed)
|
||||||
|
{
|
||||||
|
MethodInfo? method = invocation.MethodInvocationTarget ?? invocation.Method;
|
||||||
|
if (IsDryRun(method))
|
||||||
|
{
|
||||||
|
_logger.LogInformation("[DRY RUN] skipping method: {name}", method.Name);
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
|
||||||
|
await proceed(invocation, proceedInfo);
|
||||||
|
}
|
||||||
|
|
||||||
|
protected override async Task<TResult> InterceptAsync<TResult>(IInvocation invocation, IInvocationProceedInfo proceedInfo, Func<IInvocation, IInvocationProceedInfo, Task<TResult>> proceed)
|
||||||
|
{
|
||||||
|
MethodInfo? method = invocation.MethodInvocationTarget ?? invocation.Method;
|
||||||
|
if (IsDryRun(method))
|
||||||
|
{
|
||||||
|
_logger.LogInformation("[DRY RUN] skipping method: {name}", method.Name);
|
||||||
|
return default!;
|
||||||
|
}
|
||||||
|
|
||||||
|
return await proceed(invocation, proceedInfo);
|
||||||
|
}
|
||||||
|
|
||||||
|
private bool IsDryRun(MethodInfo method)
|
||||||
|
{
|
||||||
|
return method.GetCustomAttributes(typeof(DryRunSafeguardAttribute), true).Any() && _config.IsDryRun;
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,5 @@
|
|||||||
|
namespace Infrastructure.Interceptors;
|
||||||
|
|
||||||
|
public interface IDryRunService : IInterceptedService
|
||||||
|
{
|
||||||
|
}
|
||||||
@@ -0,0 +1,6 @@
|
|||||||
|
namespace Infrastructure.Interceptors;
|
||||||
|
|
||||||
|
public interface IInterceptedService
|
||||||
|
{
|
||||||
|
public object Proxy { get; set; }
|
||||||
|
}
|
||||||
@@ -0,0 +1,21 @@
|
|||||||
|
namespace Infrastructure.Interceptors;
|
||||||
|
|
||||||
|
public class InterceptedService : IInterceptedService
|
||||||
|
{
|
||||||
|
private object? _proxy;
|
||||||
|
|
||||||
|
public object Proxy
|
||||||
|
{
|
||||||
|
get
|
||||||
|
{
|
||||||
|
if (_proxy is null)
|
||||||
|
{
|
||||||
|
throw new Exception("Proxy is not set");
|
||||||
|
}
|
||||||
|
|
||||||
|
return _proxy;
|
||||||
|
}
|
||||||
|
|
||||||
|
set => _proxy = value;
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -1,3 +1,4 @@
|
|||||||
|
using Common.Attributes;
|
||||||
using Common.Configuration.Arr;
|
using Common.Configuration.Arr;
|
||||||
using Common.Configuration.Logging;
|
using Common.Configuration.Logging;
|
||||||
using Common.Configuration.QueueCleaner;
|
using Common.Configuration.QueueCleaner;
|
||||||
@@ -5,6 +6,8 @@ using Common.Helpers;
|
|||||||
using Domain.Enums;
|
using Domain.Enums;
|
||||||
using Domain.Models.Arr;
|
using Domain.Models.Arr;
|
||||||
using Domain.Models.Arr.Queue;
|
using Domain.Models.Arr.Queue;
|
||||||
|
using Infrastructure.Interceptors;
|
||||||
|
using Infrastructure.Verticals.Arr.Interfaces;
|
||||||
using Infrastructure.Verticals.ItemStriker;
|
using Infrastructure.Verticals.ItemStriker;
|
||||||
using Microsoft.Extensions.Logging;
|
using Microsoft.Extensions.Logging;
|
||||||
using Microsoft.Extensions.Options;
|
using Microsoft.Extensions.Options;
|
||||||
@@ -12,24 +15,30 @@ using Newtonsoft.Json;
|
|||||||
|
|
||||||
namespace Infrastructure.Verticals.Arr;
|
namespace Infrastructure.Verticals.Arr;
|
||||||
|
|
||||||
public abstract class ArrClient
|
public abstract class ArrClient : InterceptedService, IArrClient, IDryRunService
|
||||||
{
|
{
|
||||||
protected readonly ILogger<ArrClient> _logger;
|
protected readonly ILogger<ArrClient> _logger;
|
||||||
protected readonly HttpClient _httpClient;
|
protected readonly HttpClient _httpClient;
|
||||||
protected readonly LoggingConfig _loggingConfig;
|
protected readonly LoggingConfig _loggingConfig;
|
||||||
protected readonly QueueCleanerConfig _queueCleanerConfig;
|
protected readonly QueueCleanerConfig _queueCleanerConfig;
|
||||||
protected readonly Striker _striker;
|
protected readonly IStriker _striker;
|
||||||
|
|
||||||
|
/// <summary>
|
||||||
|
/// Constructor to be used by interceptors.
|
||||||
|
/// </summary>
|
||||||
|
protected ArrClient()
|
||||||
|
{
|
||||||
|
}
|
||||||
|
|
||||||
protected ArrClient(
|
protected ArrClient(
|
||||||
ILogger<ArrClient> logger,
|
ILogger<ArrClient> logger,
|
||||||
IHttpClientFactory httpClientFactory,
|
IHttpClientFactory httpClientFactory,
|
||||||
IOptions<LoggingConfig> loggingConfig,
|
IOptions<LoggingConfig> loggingConfig,
|
||||||
IOptions<QueueCleanerConfig> queueCleanerConfig,
|
IOptions<QueueCleanerConfig> queueCleanerConfig,
|
||||||
Striker striker
|
IStriker striker
|
||||||
)
|
)
|
||||||
{
|
{
|
||||||
_logger = logger;
|
_logger = logger;
|
||||||
_striker = striker;
|
|
||||||
_httpClient = httpClientFactory.CreateClient(Constants.HttpClientWithRetryName);
|
_httpClient = httpClientFactory.CreateClient(Constants.HttpClientWithRetryName);
|
||||||
_loggingConfig = loggingConfig.Value;
|
_loggingConfig = loggingConfig.Value;
|
||||||
_queueCleanerConfig = queueCleanerConfig.Value;
|
_queueCleanerConfig = queueCleanerConfig.Value;
|
||||||
@@ -111,14 +120,12 @@ public abstract class ArrClient
|
|||||||
{
|
{
|
||||||
Uri uri = new(arrInstance.Url, GetQueueDeleteUrlPath(record.Id, removeFromClient));
|
Uri uri = new(arrInstance.Url, GetQueueDeleteUrlPath(record.Id, removeFromClient));
|
||||||
|
|
||||||
using HttpRequestMessage request = new(HttpMethod.Delete, uri);
|
|
||||||
SetApiKey(request, arrInstance.ApiKey);
|
|
||||||
|
|
||||||
using HttpResponseMessage response = await _httpClient.SendAsync(request);
|
|
||||||
|
|
||||||
try
|
try
|
||||||
{
|
{
|
||||||
response.EnsureSuccessStatusCode();
|
using HttpRequestMessage request = new(HttpMethod.Delete, uri);
|
||||||
|
SetApiKey(request, arrInstance.ApiKey);
|
||||||
|
|
||||||
|
using var _ = await ((ArrClient)Proxy).SendRequestAsync(request);
|
||||||
|
|
||||||
_logger.LogInformation(
|
_logger.LogInformation(
|
||||||
removeFromClient
|
removeFromClient
|
||||||
@@ -157,6 +164,16 @@ public abstract class ArrClient
|
|||||||
request.Headers.Add("x-api-key", apiKey);
|
request.Headers.Add("x-api-key", apiKey);
|
||||||
}
|
}
|
||||||
|
|
||||||
|
[DryRunSafeguard]
|
||||||
|
protected virtual async Task<HttpResponseMessage> SendRequestAsync(HttpRequestMessage request)
|
||||||
|
{
|
||||||
|
HttpResponseMessage response = await _httpClient.SendAsync(request);
|
||||||
|
|
||||||
|
response.EnsureSuccessStatusCode();
|
||||||
|
|
||||||
|
return response;
|
||||||
|
}
|
||||||
|
|
||||||
private bool HasIgnoredPatterns(QueueRecord record)
|
private bool HasIgnoredPatterns(QueueRecord record)
|
||||||
{
|
{
|
||||||
if (_queueCleanerConfig.ImportFailedIgnorePatterns?.Count is null or 0)
|
if (_queueCleanerConfig.ImportFailedIgnorePatterns?.Count is null or 0)
|
||||||
|
|||||||
@@ -1,6 +1,7 @@
|
|||||||
using Common.Configuration;
|
using Common.Configuration;
|
||||||
using Common.Configuration.Arr;
|
using Common.Configuration.Arr;
|
||||||
using Domain.Models.Arr.Queue;
|
using Domain.Models.Arr.Queue;
|
||||||
|
using Infrastructure.Verticals.Arr.Interfaces;
|
||||||
using Microsoft.Extensions.Logging;
|
using Microsoft.Extensions.Logging;
|
||||||
|
|
||||||
namespace Infrastructure.Verticals.Arr;
|
namespace Infrastructure.Verticals.Arr;
|
||||||
@@ -14,7 +15,7 @@ public sealed class ArrQueueIterator
|
|||||||
_logger = logger;
|
_logger = logger;
|
||||||
}
|
}
|
||||||
|
|
||||||
public async Task Iterate(ArrClient arrClient, ArrInstance arrInstance, Func<IReadOnlyList<QueueRecord>, Task> action)
|
public async Task Iterate(IArrClient arrClient, ArrInstance arrInstance, Func<IReadOnlyList<QueueRecord>, Task> action)
|
||||||
{
|
{
|
||||||
const ushort maxPage = 100;
|
const ushort maxPage = 100;
|
||||||
ushort page = 1;
|
ushort page = 1;
|
||||||
|
|||||||
@@ -0,0 +1,19 @@
|
|||||||
|
using Common.Configuration.Arr;
|
||||||
|
using Domain.Enums;
|
||||||
|
using Domain.Models.Arr;
|
||||||
|
using Domain.Models.Arr.Queue;
|
||||||
|
|
||||||
|
namespace Infrastructure.Verticals.Arr.Interfaces;
|
||||||
|
|
||||||
|
public interface IArrClient
|
||||||
|
{
|
||||||
|
Task<QueueListResponse> GetQueueItemsAsync(ArrInstance arrInstance, int page);
|
||||||
|
|
||||||
|
Task<bool> ShouldRemoveFromQueue(InstanceType instanceType, QueueRecord record, bool isPrivateDownload);
|
||||||
|
|
||||||
|
Task DeleteQueueItemAsync(ArrInstance arrInstance, QueueRecord record, bool removeFromClient);
|
||||||
|
|
||||||
|
Task RefreshItemsAsync(ArrInstance arrInstance, HashSet<SearchItem>? items);
|
||||||
|
|
||||||
|
bool IsRecordValid(QueueRecord record);
|
||||||
|
}
|
||||||
@@ -0,0 +1,5 @@
|
|||||||
|
namespace Infrastructure.Verticals.Arr.Interfaces;
|
||||||
|
|
||||||
|
public interface ILidarrClient : IArrClient
|
||||||
|
{
|
||||||
|
}
|
||||||
@@ -0,0 +1,5 @@
|
|||||||
|
namespace Infrastructure.Verticals.Arr.Interfaces;
|
||||||
|
|
||||||
|
public interface IRadarrClient : IArrClient
|
||||||
|
{
|
||||||
|
}
|
||||||
@@ -0,0 +1,5 @@
|
|||||||
|
namespace Infrastructure.Verticals.Arr.Interfaces;
|
||||||
|
|
||||||
|
public interface ISonarrClient : IArrClient
|
||||||
|
{
|
||||||
|
}
|
||||||
@@ -5,6 +5,7 @@ using Common.Configuration.QueueCleaner;
|
|||||||
using Domain.Models.Arr;
|
using Domain.Models.Arr;
|
||||||
using Domain.Models.Arr.Queue;
|
using Domain.Models.Arr.Queue;
|
||||||
using Domain.Models.Lidarr;
|
using Domain.Models.Lidarr;
|
||||||
|
using Infrastructure.Verticals.Arr.Interfaces;
|
||||||
using Infrastructure.Verticals.ItemStriker;
|
using Infrastructure.Verticals.ItemStriker;
|
||||||
using Microsoft.Extensions.Logging;
|
using Microsoft.Extensions.Logging;
|
||||||
using Microsoft.Extensions.Options;
|
using Microsoft.Extensions.Options;
|
||||||
@@ -12,14 +13,19 @@ using Newtonsoft.Json;
|
|||||||
|
|
||||||
namespace Infrastructure.Verticals.Arr;
|
namespace Infrastructure.Verticals.Arr;
|
||||||
|
|
||||||
public sealed class LidarrClient : ArrClient
|
public class LidarrClient : ArrClient, ILidarrClient
|
||||||
{
|
{
|
||||||
|
/// <inheritdoc/>
|
||||||
|
public LidarrClient()
|
||||||
|
{
|
||||||
|
}
|
||||||
|
|
||||||
public LidarrClient(
|
public LidarrClient(
|
||||||
ILogger<LidarrClient> logger,
|
ILogger<LidarrClient> logger,
|
||||||
IHttpClientFactory httpClientFactory,
|
IHttpClientFactory httpClientFactory,
|
||||||
IOptions<LoggingConfig> loggingConfig,
|
IOptions<LoggingConfig> loggingConfig,
|
||||||
IOptions<QueueCleanerConfig> queueCleanerConfig,
|
IOptions<QueueCleanerConfig> queueCleanerConfig,
|
||||||
Striker striker
|
IStriker striker
|
||||||
) : base(logger, httpClientFactory, loggingConfig, queueCleanerConfig, striker)
|
) : base(logger, httpClientFactory, loggingConfig, queueCleanerConfig, striker)
|
||||||
{
|
{
|
||||||
}
|
}
|
||||||
@@ -54,12 +60,11 @@ public sealed class LidarrClient : ArrClient
|
|||||||
);
|
);
|
||||||
SetApiKey(request, arrInstance.ApiKey);
|
SetApiKey(request, arrInstance.ApiKey);
|
||||||
|
|
||||||
using var response = await _httpClient.SendAsync(request);
|
|
||||||
string? logContext = await ComputeCommandLogContextAsync(arrInstance, command);
|
string? logContext = await ComputeCommandLogContextAsync(arrInstance, command);
|
||||||
|
|
||||||
try
|
try
|
||||||
{
|
{
|
||||||
response.EnsureSuccessStatusCode();
|
using var _ = await ((LidarrClient)Proxy).SendRequestAsync(request);
|
||||||
|
|
||||||
_logger.LogInformation("{log}", GetSearchLog(arrInstance.Url, command, true, logContext));
|
_logger.LogInformation("{log}", GetSearchLog(arrInstance.Url, command, true, logContext));
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -5,6 +5,7 @@ using Common.Configuration.QueueCleaner;
|
|||||||
using Domain.Models.Arr;
|
using Domain.Models.Arr;
|
||||||
using Domain.Models.Arr.Queue;
|
using Domain.Models.Arr.Queue;
|
||||||
using Domain.Models.Radarr;
|
using Domain.Models.Radarr;
|
||||||
|
using Infrastructure.Verticals.Arr.Interfaces;
|
||||||
using Infrastructure.Verticals.ItemStriker;
|
using Infrastructure.Verticals.ItemStriker;
|
||||||
using Microsoft.Extensions.Logging;
|
using Microsoft.Extensions.Logging;
|
||||||
using Microsoft.Extensions.Options;
|
using Microsoft.Extensions.Options;
|
||||||
@@ -12,14 +13,19 @@ using Newtonsoft.Json;
|
|||||||
|
|
||||||
namespace Infrastructure.Verticals.Arr;
|
namespace Infrastructure.Verticals.Arr;
|
||||||
|
|
||||||
public sealed class RadarrClient : ArrClient
|
public class RadarrClient : ArrClient, IRadarrClient
|
||||||
{
|
{
|
||||||
|
/// <inheritdoc/>
|
||||||
|
public RadarrClient()
|
||||||
|
{
|
||||||
|
}
|
||||||
|
|
||||||
public RadarrClient(
|
public RadarrClient(
|
||||||
ILogger<ArrClient> logger,
|
ILogger<ArrClient> logger,
|
||||||
IHttpClientFactory httpClientFactory,
|
IHttpClientFactory httpClientFactory,
|
||||||
IOptions<LoggingConfig> loggingConfig,
|
IOptions<LoggingConfig> loggingConfig,
|
||||||
IOptions<QueueCleanerConfig> queueCleanerConfig,
|
IOptions<QueueCleanerConfig> queueCleanerConfig,
|
||||||
Striker striker
|
IStriker striker
|
||||||
) : base(logger, httpClientFactory, loggingConfig, queueCleanerConfig, striker)
|
) : base(logger, httpClientFactory, loggingConfig, queueCleanerConfig, striker)
|
||||||
{
|
{
|
||||||
}
|
}
|
||||||
@@ -62,12 +68,11 @@ public sealed class RadarrClient : ArrClient
|
|||||||
);
|
);
|
||||||
SetApiKey(request, arrInstance.ApiKey);
|
SetApiKey(request, arrInstance.ApiKey);
|
||||||
|
|
||||||
using HttpResponseMessage response = await _httpClient.SendAsync(request);
|
|
||||||
string? logContext = await ComputeCommandLogContextAsync(arrInstance, command);
|
string? logContext = await ComputeCommandLogContextAsync(arrInstance, command);
|
||||||
|
|
||||||
try
|
try
|
||||||
{
|
{
|
||||||
response.EnsureSuccessStatusCode();
|
using var _ = await ((RadarrClient)Proxy).SendRequestAsync(request);
|
||||||
|
|
||||||
_logger.LogInformation("{log}", GetSearchLog(arrInstance.Url, command, true, logContext));
|
_logger.LogInformation("{log}", GetSearchLog(arrInstance.Url, command, true, logContext));
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -5,6 +5,7 @@ using Common.Configuration.QueueCleaner;
|
|||||||
using Domain.Models.Arr;
|
using Domain.Models.Arr;
|
||||||
using Domain.Models.Arr.Queue;
|
using Domain.Models.Arr.Queue;
|
||||||
using Domain.Models.Sonarr;
|
using Domain.Models.Sonarr;
|
||||||
|
using Infrastructure.Verticals.Arr.Interfaces;
|
||||||
using Infrastructure.Verticals.ItemStriker;
|
using Infrastructure.Verticals.ItemStriker;
|
||||||
using Microsoft.Extensions.Logging;
|
using Microsoft.Extensions.Logging;
|
||||||
using Microsoft.Extensions.Options;
|
using Microsoft.Extensions.Options;
|
||||||
@@ -13,14 +14,19 @@ using Series = Domain.Models.Sonarr.Series;
|
|||||||
|
|
||||||
namespace Infrastructure.Verticals.Arr;
|
namespace Infrastructure.Verticals.Arr;
|
||||||
|
|
||||||
public sealed class SonarrClient : ArrClient
|
public class SonarrClient : ArrClient, ISonarrClient
|
||||||
{
|
{
|
||||||
|
/// <inheritdoc/>
|
||||||
|
public SonarrClient()
|
||||||
|
{
|
||||||
|
}
|
||||||
|
|
||||||
public SonarrClient(
|
public SonarrClient(
|
||||||
ILogger<SonarrClient> logger,
|
ILogger<SonarrClient> logger,
|
||||||
IHttpClientFactory httpClientFactory,
|
IHttpClientFactory httpClientFactory,
|
||||||
IOptions<LoggingConfig> loggingConfig,
|
IOptions<LoggingConfig> loggingConfig,
|
||||||
IOptions<QueueCleanerConfig> queueCleanerConfig,
|
IOptions<QueueCleanerConfig> queueCleanerConfig,
|
||||||
Striker striker
|
IStriker striker
|
||||||
) : base(logger, httpClientFactory, loggingConfig, queueCleanerConfig, striker)
|
) : base(logger, httpClientFactory, loggingConfig, queueCleanerConfig, striker)
|
||||||
{
|
{
|
||||||
}
|
}
|
||||||
@@ -58,12 +64,11 @@ public sealed class SonarrClient : ArrClient
|
|||||||
);
|
);
|
||||||
SetApiKey(request, arrInstance.ApiKey);
|
SetApiKey(request, arrInstance.ApiKey);
|
||||||
|
|
||||||
using HttpResponseMessage response = await _httpClient.SendAsync(request);
|
|
||||||
string? logContext = await ComputeCommandLogContextAsync(arrInstance, command, command.SearchType);
|
string? logContext = await ComputeCommandLogContextAsync(arrInstance, command, command.SearchType);
|
||||||
|
|
||||||
try
|
try
|
||||||
{
|
{
|
||||||
response.EnsureSuccessStatusCode();
|
using var _ = await ((SonarrClient)Proxy).SendRequestAsync(request);
|
||||||
|
|
||||||
_logger.LogInformation("{log}", GetSearchLog(command.SearchType, arrInstance.Url, command, true, logContext));
|
_logger.LogInformation("{log}", GetSearchLog(command.SearchType, arrInstance.Url, command, true, logContext));
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -7,6 +7,7 @@ using Domain.Enums;
|
|||||||
using Domain.Models.Arr;
|
using Domain.Models.Arr;
|
||||||
using Domain.Models.Arr.Queue;
|
using Domain.Models.Arr.Queue;
|
||||||
using Infrastructure.Verticals.Arr;
|
using Infrastructure.Verticals.Arr;
|
||||||
|
using Infrastructure.Verticals.Arr.Interfaces;
|
||||||
using Infrastructure.Verticals.Context;
|
using Infrastructure.Verticals.Context;
|
||||||
using Infrastructure.Verticals.DownloadClient;
|
using Infrastructure.Verticals.DownloadClient;
|
||||||
using Infrastructure.Verticals.Jobs;
|
using Infrastructure.Verticals.Jobs;
|
||||||
@@ -36,7 +37,6 @@ public sealed class ContentBlocker : GenericHandler
|
|||||||
BlocklistProvider blocklistProvider,
|
BlocklistProvider blocklistProvider,
|
||||||
DownloadServiceFactory downloadServiceFactory,
|
DownloadServiceFactory downloadServiceFactory,
|
||||||
NotificationPublisher notifier
|
NotificationPublisher notifier
|
||||||
|
|
||||||
) : base(
|
) : base(
|
||||||
logger, downloadClientConfig,
|
logger, downloadClientConfig,
|
||||||
sonarrConfig, radarrConfig, lidarrConfig,
|
sonarrConfig, radarrConfig, lidarrConfig,
|
||||||
@@ -76,7 +76,7 @@ public sealed class ContentBlocker : GenericHandler
|
|||||||
using var _ = LogContext.PushProperty("InstanceName", instanceType.ToString());
|
using var _ = LogContext.PushProperty("InstanceName", instanceType.ToString());
|
||||||
|
|
||||||
HashSet<SearchItem> itemsToBeRefreshed = [];
|
HashSet<SearchItem> itemsToBeRefreshed = [];
|
||||||
ArrClient arrClient = GetClient(instanceType);
|
IArrClient arrClient = GetClient(instanceType);
|
||||||
BlocklistType blocklistType = _blocklistProvider.GetBlocklistType(instanceType);
|
BlocklistType blocklistType = _blocklistProvider.GetBlocklistType(instanceType);
|
||||||
ConcurrentBag<string> patterns = _blocklistProvider.GetPatterns(instanceType);
|
ConcurrentBag<string> patterns = _blocklistProvider.GetPatterns(instanceType);
|
||||||
ConcurrentBag<Regex> regexes = _blocklistProvider.GetRegexes(instanceType);
|
ConcurrentBag<Regex> regexes = _blocklistProvider.GetRegexes(instanceType);
|
||||||
@@ -131,7 +131,7 @@ public sealed class ContentBlocker : GenericHandler
|
|||||||
}
|
}
|
||||||
|
|
||||||
await arrClient.DeleteQueueItemAsync(instance, record, removeFromClient);
|
await arrClient.DeleteQueueItemAsync(instance, record, removeFromClient);
|
||||||
await _notifier.NotifyQueueItemDelete(removeFromClient, DeleteReason.AllFilesBlocked);
|
await _notifier.NotifyQueueItemDeleted(removeFromClient, DeleteReason.AllFilesBlocked);
|
||||||
}
|
}
|
||||||
});
|
});
|
||||||
|
|
||||||
|
|||||||
@@ -5,7 +5,7 @@ using Microsoft.Extensions.Logging;
|
|||||||
|
|
||||||
namespace Infrastructure.Verticals.ContentBlocker;
|
namespace Infrastructure.Verticals.ContentBlocker;
|
||||||
|
|
||||||
public sealed class FilenameEvaluator
|
public class FilenameEvaluator : IFilenameEvaluator
|
||||||
{
|
{
|
||||||
private readonly ILogger<FilenameEvaluator> _logger;
|
private readonly ILogger<FilenameEvaluator> _logger;
|
||||||
|
|
||||||
@@ -31,7 +31,6 @@ public sealed class FilenameEvaluator
|
|||||||
{
|
{
|
||||||
BlocklistType.Blacklist => !patterns.Any(pattern => MatchesPattern(filename, pattern)),
|
BlocklistType.Blacklist => !patterns.Any(pattern => MatchesPattern(filename, pattern)),
|
||||||
BlocklistType.Whitelist => patterns.Any(pattern => MatchesPattern(filename, pattern)),
|
BlocklistType.Whitelist => patterns.Any(pattern => MatchesPattern(filename, pattern)),
|
||||||
_ => true
|
|
||||||
};
|
};
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -46,7 +45,6 @@ public sealed class FilenameEvaluator
|
|||||||
{
|
{
|
||||||
BlocklistType.Blacklist => !regexes.Any(regex => regex.IsMatch(filename)),
|
BlocklistType.Blacklist => !regexes.Any(regex => regex.IsMatch(filename)),
|
||||||
BlocklistType.Whitelist => regexes.Any(regex => regex.IsMatch(filename)),
|
BlocklistType.Whitelist => regexes.Any(regex => regex.IsMatch(filename)),
|
||||||
_ => true
|
|
||||||
};
|
};
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -76,6 +74,6 @@ public sealed class FilenameEvaluator
|
|||||||
);
|
);
|
||||||
}
|
}
|
||||||
|
|
||||||
return filename == pattern;
|
return filename.Equals(pattern, StringComparison.InvariantCultureIgnoreCase);
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
@@ -0,0 +1,10 @@
|
|||||||
|
using System.Collections.Concurrent;
|
||||||
|
using System.Text.RegularExpressions;
|
||||||
|
using Common.Configuration.ContentBlocker;
|
||||||
|
|
||||||
|
namespace Infrastructure.Verticals.ContentBlocker;
|
||||||
|
|
||||||
|
public interface IFilenameEvaluator
|
||||||
|
{
|
||||||
|
bool IsValid(string filename, BlocklistType type, ConcurrentBag<string> patterns, ConcurrentBag<Regex> regexes);
|
||||||
|
}
|
||||||
@@ -17,8 +17,8 @@ public static class ContextProvider
|
|||||||
return _asyncLocalDict.Value?.TryGetValue(key, out object? value) is true ? value : null;
|
return _asyncLocalDict.Value?.TryGetValue(key, out object? value) is true ? value : null;
|
||||||
}
|
}
|
||||||
|
|
||||||
public static T? Get<T>(string key) where T : class
|
public static T Get<T>(string key) where T : class
|
||||||
{
|
{
|
||||||
return Get(key) as T;
|
return Get(key) as T ?? throw new Exception($"failed to get \"{key}\" from context");
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -0,0 +1,99 @@
|
|||||||
|
using Common.Configuration.Arr;
|
||||||
|
using Common.Configuration.DownloadCleaner;
|
||||||
|
using Common.Configuration.DownloadClient;
|
||||||
|
using Domain.Enums;
|
||||||
|
using Domain.Models.Arr.Queue;
|
||||||
|
using Infrastructure.Verticals.Arr;
|
||||||
|
using Infrastructure.Verticals.Arr.Interfaces;
|
||||||
|
using Infrastructure.Verticals.DownloadClient;
|
||||||
|
using Infrastructure.Verticals.Jobs;
|
||||||
|
using Infrastructure.Verticals.Notifications;
|
||||||
|
using Microsoft.Extensions.Logging;
|
||||||
|
using Microsoft.Extensions.Options;
|
||||||
|
using Serilog.Context;
|
||||||
|
|
||||||
|
namespace Infrastructure.Verticals.DownloadCleaner;
|
||||||
|
|
||||||
|
public sealed class DownloadCleaner : GenericHandler
|
||||||
|
{
|
||||||
|
private readonly DownloadCleanerConfig _config;
|
||||||
|
private readonly HashSet<string> _excludedHashes = [];
|
||||||
|
|
||||||
|
public DownloadCleaner(
|
||||||
|
ILogger<DownloadCleaner> logger,
|
||||||
|
IOptions<DownloadCleanerConfig> config,
|
||||||
|
IOptions<DownloadClientConfig> downloadClientConfig,
|
||||||
|
IOptions<SonarrConfig> sonarrConfig,
|
||||||
|
IOptions<RadarrConfig> radarrConfig,
|
||||||
|
IOptions<LidarrConfig> lidarrConfig,
|
||||||
|
SonarrClient sonarrClient,
|
||||||
|
RadarrClient radarrClient,
|
||||||
|
LidarrClient lidarrClient,
|
||||||
|
ArrQueueIterator arrArrQueueIterator,
|
||||||
|
DownloadServiceFactory downloadServiceFactory,
|
||||||
|
NotificationPublisher notifier
|
||||||
|
) : base(
|
||||||
|
logger, downloadClientConfig,
|
||||||
|
sonarrConfig, radarrConfig, lidarrConfig,
|
||||||
|
sonarrClient, radarrClient, lidarrClient,
|
||||||
|
arrArrQueueIterator, downloadServiceFactory,
|
||||||
|
notifier
|
||||||
|
)
|
||||||
|
{
|
||||||
|
_config = config.Value;
|
||||||
|
_config.Validate();
|
||||||
|
}
|
||||||
|
|
||||||
|
public override async Task ExecuteAsync()
|
||||||
|
{
|
||||||
|
if (_config.Categories?.Count is null or 0)
|
||||||
|
{
|
||||||
|
_logger.LogWarning("no categories configured");
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
|
||||||
|
await _downloadService.LoginAsync();
|
||||||
|
|
||||||
|
List<object>? downloads = await _downloadService.GetAllDownloadsToBeCleaned(_config.Categories);
|
||||||
|
|
||||||
|
if (downloads?.Count is null or 0)
|
||||||
|
{
|
||||||
|
_logger.LogDebug("no downloads found in the download client");
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
|
||||||
|
// wait for the downloads to appear in the arr queue
|
||||||
|
await Task.Delay(10 * 1000);
|
||||||
|
|
||||||
|
await ProcessArrConfigAsync(_sonarrConfig, InstanceType.Sonarr, true);
|
||||||
|
await ProcessArrConfigAsync(_radarrConfig, InstanceType.Radarr, true);
|
||||||
|
await ProcessArrConfigAsync(_lidarrConfig, InstanceType.Lidarr, true);
|
||||||
|
|
||||||
|
await _downloadService.CleanDownloads(downloads, _config.Categories, _excludedHashes);
|
||||||
|
}
|
||||||
|
|
||||||
|
protected override async Task ProcessInstanceAsync(ArrInstance instance, InstanceType instanceType)
|
||||||
|
{
|
||||||
|
using var _ = LogContext.PushProperty("InstanceName", instanceType.ToString());
|
||||||
|
|
||||||
|
IArrClient arrClient = GetClient(instanceType);
|
||||||
|
|
||||||
|
await _arrArrQueueIterator.Iterate(arrClient, instance, async items =>
|
||||||
|
{
|
||||||
|
var groups = items
|
||||||
|
.Where(x => !string.IsNullOrEmpty(x.DownloadId))
|
||||||
|
.GroupBy(x => x.DownloadId)
|
||||||
|
.ToList();
|
||||||
|
|
||||||
|
foreach (QueueRecord record in groups.Select(group => group.First()))
|
||||||
|
{
|
||||||
|
_excludedHashes.Add(record.DownloadId.ToLowerInvariant());
|
||||||
|
}
|
||||||
|
});
|
||||||
|
}
|
||||||
|
|
||||||
|
public override void Dispose()
|
||||||
|
{
|
||||||
|
_downloadService.Dispose();
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -63,6 +63,26 @@ public sealed class DelugeClient
|
|||||||
return torrents.FirstOrDefault();
|
return torrents.FirstOrDefault();
|
||||||
}
|
}
|
||||||
|
|
||||||
|
public async Task<TorrentStatus?> GetTorrentStatus(string hash)
|
||||||
|
{
|
||||||
|
return await SendRequest<TorrentStatus?>(
|
||||||
|
"web.get_torrent_status",
|
||||||
|
hash,
|
||||||
|
new[] { "hash", "state", "name", "eta", "private", "total_done", "label", "seeding_time", "ratio" }
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
public async Task<List<TorrentStatus>?> GetStatusForAllTorrents()
|
||||||
|
{
|
||||||
|
Dictionary<string, TorrentStatus>? downloads = await SendRequest<Dictionary<string, TorrentStatus>?>(
|
||||||
|
"core.get_torrents_status",
|
||||||
|
"",
|
||||||
|
new[] { "hash", "state", "name", "eta", "private", "total_done", "label", "seeding_time", "ratio" }
|
||||||
|
);
|
||||||
|
|
||||||
|
return downloads?.Values.ToList();
|
||||||
|
}
|
||||||
|
|
||||||
public async Task<DelugeContents?> GetTorrentFiles(string hash)
|
public async Task<DelugeContents?> GetTorrentFiles(string hash)
|
||||||
{
|
{
|
||||||
return await SendRequest<DelugeContents?>("web.get_torrent_files", hash);
|
return await SendRequest<DelugeContents?>("web.get_torrent_files", hash);
|
||||||
@@ -78,9 +98,9 @@ public sealed class DelugeClient
|
|||||||
await SendRequest<DelugeResponse<object>>("core.set_torrent_options", hash, filePriorities);
|
await SendRequest<DelugeResponse<object>>("core.set_torrent_options", hash, filePriorities);
|
||||||
}
|
}
|
||||||
|
|
||||||
public async Task<DelugeResponse<object>> DeleteTorrent(string hash)
|
public async Task DeleteTorrents(List<string> hashes)
|
||||||
{
|
{
|
||||||
return await SendRequest<DelugeResponse<object>>("core.remove_torrents", new List<string> { hash }, true);
|
await SendRequest<DelugeResponse<object>>("core.remove_torrents", hashes, true);
|
||||||
}
|
}
|
||||||
|
|
||||||
private async Task<String> PostJson(String json)
|
private async Task<String> PostJson(String json)
|
||||||
|
|||||||
@@ -1,32 +1,44 @@
|
|||||||
using System.Collections.Concurrent;
|
using System.Collections.Concurrent;
|
||||||
using System.Text.RegularExpressions;
|
using System.Text.RegularExpressions;
|
||||||
|
using Common.Attributes;
|
||||||
using Common.Configuration.ContentBlocker;
|
using Common.Configuration.ContentBlocker;
|
||||||
|
using Common.Configuration.DownloadCleaner;
|
||||||
using Common.Configuration.DownloadClient;
|
using Common.Configuration.DownloadClient;
|
||||||
using Common.Configuration.QueueCleaner;
|
using Common.Configuration.QueueCleaner;
|
||||||
using Domain.Enums;
|
using Domain.Enums;
|
||||||
using Domain.Models.Deluge.Response;
|
using Domain.Models.Deluge.Response;
|
||||||
using Infrastructure.Verticals.ContentBlocker;
|
using Infrastructure.Verticals.ContentBlocker;
|
||||||
|
using Infrastructure.Verticals.Context;
|
||||||
using Infrastructure.Verticals.ItemStriker;
|
using Infrastructure.Verticals.ItemStriker;
|
||||||
|
using Infrastructure.Verticals.Notifications;
|
||||||
|
using MassTransit.Configuration;
|
||||||
using Microsoft.Extensions.Caching.Memory;
|
using Microsoft.Extensions.Caching.Memory;
|
||||||
using Microsoft.Extensions.Logging;
|
using Microsoft.Extensions.Logging;
|
||||||
using Microsoft.Extensions.Options;
|
using Microsoft.Extensions.Options;
|
||||||
|
|
||||||
namespace Infrastructure.Verticals.DownloadClient.Deluge;
|
namespace Infrastructure.Verticals.DownloadClient.Deluge;
|
||||||
|
|
||||||
public sealed class DelugeService : DownloadServiceBase
|
public class DelugeService : DownloadService, IDelugeService
|
||||||
{
|
{
|
||||||
private readonly DelugeClient _client;
|
private readonly DelugeClient _client;
|
||||||
|
|
||||||
|
/// <inheritdoc/>
|
||||||
|
public DelugeService()
|
||||||
|
{
|
||||||
|
}
|
||||||
|
|
||||||
public DelugeService(
|
public DelugeService(
|
||||||
ILogger<DelugeService> logger,
|
ILogger<DelugeService> logger,
|
||||||
IOptions<DelugeConfig> config,
|
IOptions<DelugeConfig> config,
|
||||||
IHttpClientFactory httpClientFactory,
|
IHttpClientFactory httpClientFactory,
|
||||||
IOptions<QueueCleanerConfig> queueCleanerConfig,
|
IOptions<QueueCleanerConfig> queueCleanerConfig,
|
||||||
IOptions<ContentBlockerConfig> contentBlockerConfig,
|
IOptions<ContentBlockerConfig> contentBlockerConfig,
|
||||||
|
IOptions<DownloadCleanerConfig> downloadCleanerConfig,
|
||||||
IMemoryCache cache,
|
IMemoryCache cache,
|
||||||
FilenameEvaluator filenameEvaluator,
|
IFilenameEvaluator filenameEvaluator,
|
||||||
Striker striker
|
IStriker striker,
|
||||||
) : base(logger, queueCleanerConfig, contentBlockerConfig, cache, filenameEvaluator, striker)
|
NotificationPublisher notifier
|
||||||
|
) : base(logger, queueCleanerConfig, contentBlockerConfig, downloadCleanerConfig, cache, filenameEvaluator, striker, notifier)
|
||||||
{
|
{
|
||||||
config.Value.Validate();
|
config.Value.Validate();
|
||||||
_client = new (config, httpClientFactory);
|
_client = new (config, httpClientFactory);
|
||||||
@@ -45,7 +57,7 @@ public sealed class DelugeService : DownloadServiceBase
|
|||||||
DelugeContents? contents = null;
|
DelugeContents? contents = null;
|
||||||
StalledResult result = new();
|
StalledResult result = new();
|
||||||
|
|
||||||
TorrentStatus? status = await GetTorrentStatus(hash);
|
TorrentStatus? status = await _client.GetTorrentStatus(hash);
|
||||||
|
|
||||||
if (status?.Hash is null)
|
if (status?.Hash is null)
|
||||||
{
|
{
|
||||||
@@ -98,7 +110,7 @@ public sealed class DelugeService : DownloadServiceBase
|
|||||||
{
|
{
|
||||||
hash = hash.ToLowerInvariant();
|
hash = hash.ToLowerInvariant();
|
||||||
|
|
||||||
TorrentStatus? status = await GetTorrentStatus(hash);
|
TorrentStatus? status = await _client.GetTorrentStatus(hash);
|
||||||
BlockFilesResult result = new();
|
BlockFilesResult result = new();
|
||||||
|
|
||||||
if (status?.Hash is null)
|
if (status?.Hash is null)
|
||||||
@@ -178,17 +190,89 @@ public sealed class DelugeService : DownloadServiceBase
|
|||||||
return result;
|
return result;
|
||||||
}
|
}
|
||||||
|
|
||||||
await _client.ChangeFilesPriority(hash, sortedPriorities);
|
await ((DelugeService)Proxy).ChangeFilesPriority(hash, sortedPriorities);
|
||||||
|
|
||||||
return result;
|
return result;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
public override async Task<List<object>?> GetAllDownloadsToBeCleaned(List<Category> categories)
|
||||||
|
{
|
||||||
|
return (await _client.GetStatusForAllTorrents())
|
||||||
|
?.Where(x => !string.IsNullOrEmpty(x.Hash))
|
||||||
|
.Where(x => x.State?.Equals("seeding", StringComparison.InvariantCultureIgnoreCase) is true)
|
||||||
|
.Where(x => categories.Any(cat => cat.Name.Equals(x.Label, StringComparison.InvariantCultureIgnoreCase)))
|
||||||
|
.Cast<object>()
|
||||||
|
.ToList();
|
||||||
|
}
|
||||||
|
|
||||||
/// <inheritdoc/>
|
/// <inheritdoc/>
|
||||||
public override async Task Delete(string hash)
|
public override async Task CleanDownloads(List<object> downloads, List<Category> categoriesToClean, HashSet<string> excludedHashes)
|
||||||
|
{
|
||||||
|
foreach (TorrentStatus download in downloads)
|
||||||
|
{
|
||||||
|
if (string.IsNullOrEmpty(download.Hash))
|
||||||
|
{
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
|
||||||
|
Category? category = categoriesToClean
|
||||||
|
.FirstOrDefault(x => x.Name.Equals(download.Label, StringComparison.InvariantCultureIgnoreCase));
|
||||||
|
|
||||||
|
if (category is null)
|
||||||
|
{
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
|
||||||
|
if (excludedHashes.Any(x => x.Equals(download.Hash, StringComparison.InvariantCultureIgnoreCase)))
|
||||||
|
{
|
||||||
|
_logger.LogDebug("skip | download is used by an arr | {name}", download.Name);
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
|
||||||
|
if (!_downloadCleanerConfig.DeletePrivate && download.Private)
|
||||||
|
{
|
||||||
|
_logger.LogDebug("skip | download is private | {name}", download.Name);
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
|
||||||
|
ContextProvider.Set("downloadName", download.Name);
|
||||||
|
ContextProvider.Set("hash", download.Hash);
|
||||||
|
|
||||||
|
TimeSpan seedingTime = TimeSpan.FromSeconds(download.SeedingTime);
|
||||||
|
SeedingCheckResult result = ShouldCleanDownload(download.Ratio, seedingTime, category);
|
||||||
|
|
||||||
|
if (!result.ShouldClean)
|
||||||
|
{
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
|
||||||
|
await ((DelugeService)Proxy).DeleteDownload(download.Hash);
|
||||||
|
|
||||||
|
_logger.LogInformation(
|
||||||
|
"download cleaned | {reason} reached | {name}",
|
||||||
|
result.Reason is CleanReason.MaxRatioReached
|
||||||
|
? "MAX_RATIO & MIN_SEED_TIME"
|
||||||
|
: "MAX_SEED_TIME",
|
||||||
|
download.Name
|
||||||
|
);
|
||||||
|
|
||||||
|
await _notifier.NotifyDownloadCleaned(download.Ratio, seedingTime, category.Name, result.Reason);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// <inheritdoc/>
|
||||||
|
[DryRunSafeguard]
|
||||||
|
public override async Task DeleteDownload(string hash)
|
||||||
{
|
{
|
||||||
hash = hash.ToLowerInvariant();
|
hash = hash.ToLowerInvariant();
|
||||||
|
|
||||||
await _client.DeleteTorrent(hash);
|
await _client.DeleteTorrents([hash]);
|
||||||
|
}
|
||||||
|
|
||||||
|
[DryRunSafeguard]
|
||||||
|
protected virtual async Task ChangeFilesPriority(string hash, List<int> sortedPriorities)
|
||||||
|
{
|
||||||
|
await _client.ChangeFilesPriority(hash, sortedPriorities);
|
||||||
}
|
}
|
||||||
|
|
||||||
private async Task<bool> IsItemStuckAndShouldRemove(TorrentStatus status)
|
private async Task<bool> IsItemStuckAndShouldRemove(TorrentStatus status)
|
||||||
@@ -220,15 +304,6 @@ public sealed class DelugeService : DownloadServiceBase
|
|||||||
return await StrikeAndCheckLimit(status.Hash!, status.Name!);
|
return await StrikeAndCheckLimit(status.Hash!, status.Name!);
|
||||||
}
|
}
|
||||||
|
|
||||||
private async Task<TorrentStatus?> GetTorrentStatus(string hash)
|
|
||||||
{
|
|
||||||
return await _client.SendRequest<TorrentStatus?>(
|
|
||||||
"web.get_torrent_status",
|
|
||||||
hash,
|
|
||||||
new[] { "hash", "state", "name", "eta", "private", "total_done" }
|
|
||||||
);
|
|
||||||
}
|
|
||||||
|
|
||||||
private static void ProcessFiles(Dictionary<string, DelugeFileOrDirectory>? contents, Action<string, DelugeFileOrDirectory> processFile)
|
private static void ProcessFiles(Dictionary<string, DelugeFileOrDirectory>? contents, Action<string, DelugeFileOrDirectory> processFile)
|
||||||
{
|
{
|
||||||
if (contents is null)
|
if (contents is null)
|
||||||
|
|||||||
@@ -0,0 +1,5 @@
|
|||||||
|
namespace Infrastructure.Verticals.DownloadClient.Deluge;
|
||||||
|
|
||||||
|
public interface IDelugeService : IDownloadService
|
||||||
|
{
|
||||||
|
}
|
||||||
@@ -0,0 +1,183 @@
|
|||||||
|
using System.Collections.Concurrent;
|
||||||
|
using System.Text.RegularExpressions;
|
||||||
|
using Common.Configuration.ContentBlocker;
|
||||||
|
using Common.Configuration.DownloadCleaner;
|
||||||
|
using Common.Configuration.QueueCleaner;
|
||||||
|
using Common.Helpers;
|
||||||
|
using Domain.Enums;
|
||||||
|
using Domain.Models.Cache;
|
||||||
|
using Infrastructure.Helpers;
|
||||||
|
using Infrastructure.Interceptors;
|
||||||
|
using Infrastructure.Verticals.ContentBlocker;
|
||||||
|
using Infrastructure.Verticals.Context;
|
||||||
|
using Infrastructure.Verticals.ItemStriker;
|
||||||
|
using Infrastructure.Verticals.Notifications;
|
||||||
|
using Microsoft.Extensions.Caching.Memory;
|
||||||
|
using Microsoft.Extensions.Logging;
|
||||||
|
using Microsoft.Extensions.Options;
|
||||||
|
|
||||||
|
namespace Infrastructure.Verticals.DownloadClient;
|
||||||
|
|
||||||
|
public abstract class DownloadService : InterceptedService, IDownloadService
|
||||||
|
{
|
||||||
|
protected readonly ILogger<DownloadService> _logger;
|
||||||
|
protected readonly QueueCleanerConfig _queueCleanerConfig;
|
||||||
|
protected readonly ContentBlockerConfig _contentBlockerConfig;
|
||||||
|
protected readonly DownloadCleanerConfig _downloadCleanerConfig;
|
||||||
|
protected readonly IMemoryCache _cache;
|
||||||
|
protected readonly IFilenameEvaluator _filenameEvaluator;
|
||||||
|
protected readonly IStriker _striker;
|
||||||
|
protected readonly MemoryCacheEntryOptions _cacheOptions;
|
||||||
|
protected readonly NotificationPublisher _notifier;
|
||||||
|
|
||||||
|
/// <summary>
|
||||||
|
/// Constructor to be used by interceptors.
|
||||||
|
/// </summary>
|
||||||
|
protected DownloadService()
|
||||||
|
{
|
||||||
|
}
|
||||||
|
|
||||||
|
protected DownloadService(
|
||||||
|
ILogger<DownloadService> logger,
|
||||||
|
IOptions<QueueCleanerConfig> queueCleanerConfig,
|
||||||
|
IOptions<ContentBlockerConfig> contentBlockerConfig,
|
||||||
|
IOptions<DownloadCleanerConfig> downloadCleanerConfig,
|
||||||
|
IMemoryCache cache,
|
||||||
|
IFilenameEvaluator filenameEvaluator,
|
||||||
|
IStriker striker,
|
||||||
|
NotificationPublisher notifier)
|
||||||
|
{
|
||||||
|
_logger = logger;
|
||||||
|
_queueCleanerConfig = queueCleanerConfig.Value;
|
||||||
|
_contentBlockerConfig = contentBlockerConfig.Value;
|
||||||
|
_downloadCleanerConfig = downloadCleanerConfig.Value;
|
||||||
|
_cache = cache;
|
||||||
|
_filenameEvaluator = filenameEvaluator;
|
||||||
|
_striker = striker;
|
||||||
|
_notifier = notifier;
|
||||||
|
_cacheOptions = new MemoryCacheEntryOptions()
|
||||||
|
.SetSlidingExpiration(StaticConfiguration.TriggerValue + Constants.CacheLimitBuffer);
|
||||||
|
}
|
||||||
|
|
||||||
|
public abstract void Dispose();
|
||||||
|
|
||||||
|
public abstract Task LoginAsync();
|
||||||
|
|
||||||
|
public abstract Task<StalledResult> ShouldRemoveFromArrQueueAsync(string hash);
|
||||||
|
|
||||||
|
/// <inheritdoc/>
|
||||||
|
public abstract Task<BlockFilesResult> BlockUnwantedFilesAsync(
|
||||||
|
string hash,
|
||||||
|
BlocklistType blocklistType,
|
||||||
|
ConcurrentBag<string> patterns,
|
||||||
|
ConcurrentBag<Regex> regexes
|
||||||
|
);
|
||||||
|
|
||||||
|
/// <inheritdoc/>
|
||||||
|
public abstract Task DeleteDownload(string hash);
|
||||||
|
|
||||||
|
/// <inheritdoc/>
|
||||||
|
public abstract Task<List<object>?> GetAllDownloadsToBeCleaned(List<Category> categories);
|
||||||
|
|
||||||
|
/// <inheritdoc/>
|
||||||
|
public abstract Task CleanDownloads(List<object> downloads, List<Category> categoriesToClean, HashSet<string> excludedHashes);
|
||||||
|
|
||||||
|
protected void ResetStrikesOnProgress(string hash, long downloaded)
|
||||||
|
{
|
||||||
|
if (!_queueCleanerConfig.StalledResetStrikesOnProgress)
|
||||||
|
{
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
|
||||||
|
if (_cache.TryGetValue(CacheKeys.Item(hash), out CacheItem? cachedItem) && cachedItem is not null && downloaded > cachedItem.Downloaded)
|
||||||
|
{
|
||||||
|
// cache item found
|
||||||
|
_cache.Remove(CacheKeys.Strike(StrikeType.Stalled, hash));
|
||||||
|
_logger.LogDebug("resetting strikes for {hash} due to progress", hash);
|
||||||
|
}
|
||||||
|
|
||||||
|
_cache.Set(CacheKeys.Item(hash), new CacheItem { Downloaded = downloaded }, _cacheOptions);
|
||||||
|
}
|
||||||
|
|
||||||
|
/// <summary>
|
||||||
|
/// Strikes an item and checks if the limit has been reached.
|
||||||
|
/// </summary>
|
||||||
|
/// <param name="hash">The torrent hash.</param>
|
||||||
|
/// <param name="itemName">The name or title of the item.</param>
|
||||||
|
/// <returns>True if the limit has been reached; otherwise, false.</returns>
|
||||||
|
protected async Task<bool> StrikeAndCheckLimit(string hash, string itemName)
|
||||||
|
{
|
||||||
|
return await _striker.StrikeAndCheckLimit(hash, itemName, _queueCleanerConfig.StalledMaxStrikes, StrikeType.Stalled);
|
||||||
|
}
|
||||||
|
|
||||||
|
protected SeedingCheckResult ShouldCleanDownload(double ratio, TimeSpan seedingTime, Category category)
|
||||||
|
{
|
||||||
|
// check ratio
|
||||||
|
if (DownloadReachedRatio(ratio, seedingTime, category))
|
||||||
|
{
|
||||||
|
return new()
|
||||||
|
{
|
||||||
|
ShouldClean = true,
|
||||||
|
Reason = CleanReason.MaxRatioReached
|
||||||
|
};
|
||||||
|
}
|
||||||
|
|
||||||
|
// check max seed time
|
||||||
|
if (DownloadReachedMaxSeedTime(seedingTime, category))
|
||||||
|
{
|
||||||
|
return new()
|
||||||
|
{
|
||||||
|
ShouldClean = true,
|
||||||
|
Reason = CleanReason.MaxSeedTimeReached
|
||||||
|
};
|
||||||
|
}
|
||||||
|
|
||||||
|
return new();
|
||||||
|
}
|
||||||
|
|
||||||
|
private bool DownloadReachedRatio(double ratio, TimeSpan seedingTime, Category category)
|
||||||
|
{
|
||||||
|
if (category.MaxRatio < 0)
|
||||||
|
{
|
||||||
|
return false;
|
||||||
|
}
|
||||||
|
|
||||||
|
string downloadName = ContextProvider.Get<string>("downloadName");
|
||||||
|
TimeSpan minSeedingTime = TimeSpan.FromHours(category.MinSeedTime);
|
||||||
|
|
||||||
|
if (category.MinSeedTime > 0 && seedingTime < minSeedingTime)
|
||||||
|
{
|
||||||
|
_logger.LogDebug("skip | download has not reached MIN_SEED_TIME | {name}", downloadName);
|
||||||
|
return false;
|
||||||
|
}
|
||||||
|
|
||||||
|
if (ratio < category.MaxRatio)
|
||||||
|
{
|
||||||
|
_logger.LogDebug("skip | download has not reached MAX_RATIO | {name}", downloadName);
|
||||||
|
return false;
|
||||||
|
}
|
||||||
|
|
||||||
|
// max ration is 0 or reached
|
||||||
|
return true;
|
||||||
|
}
|
||||||
|
|
||||||
|
private bool DownloadReachedMaxSeedTime(TimeSpan seedingTime, Category category)
|
||||||
|
{
|
||||||
|
if (category.MaxSeedTime < 0)
|
||||||
|
{
|
||||||
|
return false;
|
||||||
|
}
|
||||||
|
|
||||||
|
string downloadName = ContextProvider.Get<string>("downloadName");
|
||||||
|
TimeSpan maxSeedingTime = TimeSpan.FromHours(category.MaxSeedTime);
|
||||||
|
|
||||||
|
if (category.MaxSeedTime > 0 && seedingTime < maxSeedingTime)
|
||||||
|
{
|
||||||
|
_logger.LogDebug("skip | download has not reached MAX_SEED_TIME | {name}", downloadName);
|
||||||
|
return false;
|
||||||
|
}
|
||||||
|
|
||||||
|
// max seed time is 0 or reached
|
||||||
|
return true;
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -1,90 +0,0 @@
|
|||||||
using System.Collections.Concurrent;
|
|
||||||
using System.Text.RegularExpressions;
|
|
||||||
using Common.Configuration.ContentBlocker;
|
|
||||||
using Common.Configuration.QueueCleaner;
|
|
||||||
using Common.Helpers;
|
|
||||||
using Domain.Enums;
|
|
||||||
using Domain.Models.Cache;
|
|
||||||
using Infrastructure.Helpers;
|
|
||||||
using Infrastructure.Verticals.ContentBlocker;
|
|
||||||
using Infrastructure.Verticals.ItemStriker;
|
|
||||||
using Microsoft.Extensions.Caching.Memory;
|
|
||||||
using Microsoft.Extensions.Logging;
|
|
||||||
using Microsoft.Extensions.Options;
|
|
||||||
|
|
||||||
namespace Infrastructure.Verticals.DownloadClient;
|
|
||||||
|
|
||||||
public abstract class DownloadServiceBase : IDownloadService
|
|
||||||
{
|
|
||||||
protected readonly ILogger<DownloadServiceBase> _logger;
|
|
||||||
protected readonly QueueCleanerConfig _queueCleanerConfig;
|
|
||||||
protected readonly ContentBlockerConfig _contentBlockerConfig;
|
|
||||||
protected readonly IMemoryCache _cache;
|
|
||||||
protected readonly FilenameEvaluator _filenameEvaluator;
|
|
||||||
protected readonly Striker _striker;
|
|
||||||
protected readonly MemoryCacheEntryOptions _cacheOptions;
|
|
||||||
|
|
||||||
protected DownloadServiceBase(
|
|
||||||
ILogger<DownloadServiceBase> logger,
|
|
||||||
IOptions<QueueCleanerConfig> queueCleanerConfig,
|
|
||||||
IOptions<ContentBlockerConfig> contentBlockerConfig,
|
|
||||||
IMemoryCache cache,
|
|
||||||
FilenameEvaluator filenameEvaluator,
|
|
||||||
Striker striker
|
|
||||||
)
|
|
||||||
{
|
|
||||||
_logger = logger;
|
|
||||||
_queueCleanerConfig = queueCleanerConfig.Value;
|
|
||||||
_contentBlockerConfig = contentBlockerConfig.Value;
|
|
||||||
_cache = cache;
|
|
||||||
_filenameEvaluator = filenameEvaluator;
|
|
||||||
_striker = striker;
|
|
||||||
_cacheOptions = new MemoryCacheEntryOptions()
|
|
||||||
.SetSlidingExpiration(StaticConfiguration.TriggerValue + Constants.CacheLimitBuffer);
|
|
||||||
}
|
|
||||||
|
|
||||||
public abstract void Dispose();
|
|
||||||
|
|
||||||
public abstract Task LoginAsync();
|
|
||||||
|
|
||||||
public abstract Task<StalledResult> ShouldRemoveFromArrQueueAsync(string hash);
|
|
||||||
|
|
||||||
/// <inheritdoc/>
|
|
||||||
public abstract Task<BlockFilesResult> BlockUnwantedFilesAsync(
|
|
||||||
string hash,
|
|
||||||
BlocklistType blocklistType,
|
|
||||||
ConcurrentBag<string> patterns,
|
|
||||||
ConcurrentBag<Regex> regexes
|
|
||||||
);
|
|
||||||
|
|
||||||
/// <inheritdoc/>
|
|
||||||
public abstract Task Delete(string hash);
|
|
||||||
|
|
||||||
protected void ResetStrikesOnProgress(string hash, long downloaded)
|
|
||||||
{
|
|
||||||
if (!_queueCleanerConfig.StalledResetStrikesOnProgress)
|
|
||||||
{
|
|
||||||
return;
|
|
||||||
}
|
|
||||||
|
|
||||||
if (_cache.TryGetValue(CacheKeys.Item(hash), out CacheItem? cachedItem) && cachedItem is not null && downloaded > cachedItem.Downloaded)
|
|
||||||
{
|
|
||||||
// cache item found
|
|
||||||
_cache.Remove(CacheKeys.Strike(StrikeType.Stalled, hash));
|
|
||||||
_logger.LogDebug("resetting strikes for {hash} due to progress", hash);
|
|
||||||
}
|
|
||||||
|
|
||||||
_cache.Set(CacheKeys.Item(hash), new CacheItem { Downloaded = downloaded }, _cacheOptions);
|
|
||||||
}
|
|
||||||
|
|
||||||
/// <summary>
|
|
||||||
/// Strikes an item and checks if the limit has been reached.
|
|
||||||
/// </summary>
|
|
||||||
/// <param name="hash">The torrent hash.</param>
|
|
||||||
/// <param name="itemName">The name or title of the item.</param>
|
|
||||||
/// <returns>True if the limit has been reached; otherwise, false.</returns>
|
|
||||||
protected async Task<bool> StrikeAndCheckLimit(string hash, string itemName)
|
|
||||||
{
|
|
||||||
return await _striker.StrikeAndCheckLimit(hash, itemName, _queueCleanerConfig.StalledMaxStrikes, StrikeType.Stalled);
|
|
||||||
}
|
|
||||||
}
|
|
||||||
@@ -1,18 +1,20 @@
|
|||||||
using System.Collections.Concurrent;
|
using System.Collections.Concurrent;
|
||||||
using System.Text.RegularExpressions;
|
using System.Text.RegularExpressions;
|
||||||
using Common.Configuration.ContentBlocker;
|
using Common.Configuration.ContentBlocker;
|
||||||
|
using Common.Configuration.DownloadCleaner;
|
||||||
using Common.Configuration.QueueCleaner;
|
using Common.Configuration.QueueCleaner;
|
||||||
using Infrastructure.Verticals.ContentBlocker;
|
using Infrastructure.Verticals.ContentBlocker;
|
||||||
using Infrastructure.Verticals.ItemStriker;
|
using Infrastructure.Verticals.ItemStriker;
|
||||||
|
using Infrastructure.Verticals.Notifications;
|
||||||
using Microsoft.Extensions.Caching.Memory;
|
using Microsoft.Extensions.Caching.Memory;
|
||||||
using Microsoft.Extensions.Logging;
|
using Microsoft.Extensions.Logging;
|
||||||
using Microsoft.Extensions.Options;
|
using Microsoft.Extensions.Options;
|
||||||
|
|
||||||
namespace Infrastructure.Verticals.DownloadClient;
|
namespace Infrastructure.Verticals.DownloadClient;
|
||||||
|
|
||||||
public sealed class DummyDownloadService : DownloadServiceBase
|
public sealed class DummyDownloadService : DownloadService
|
||||||
{
|
{
|
||||||
public DummyDownloadService(ILogger<DownloadServiceBase> logger, IOptions<QueueCleanerConfig> queueCleanerConfig, IOptions<ContentBlockerConfig> contentBlockerConfig, IMemoryCache cache, FilenameEvaluator filenameEvaluator, Striker striker) : base(logger, queueCleanerConfig, contentBlockerConfig, cache, filenameEvaluator, striker)
|
public DummyDownloadService(ILogger<DownloadService> logger, IOptions<QueueCleanerConfig> queueCleanerConfig, IOptions<ContentBlockerConfig> contentBlockerConfig, IOptions<DownloadCleanerConfig> downloadCleanerConfig, IMemoryCache cache, IFilenameEvaluator filenameEvaluator, IStriker striker, NotificationPublisher notifier) : base(logger, queueCleanerConfig, contentBlockerConfig, downloadCleanerConfig, cache, filenameEvaluator, striker, notifier)
|
||||||
{
|
{
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -35,7 +37,17 @@ public sealed class DummyDownloadService : DownloadServiceBase
|
|||||||
throw new NotImplementedException();
|
throw new NotImplementedException();
|
||||||
}
|
}
|
||||||
|
|
||||||
public override Task Delete(string hash)
|
public override Task<List<object>?> GetAllDownloadsToBeCleaned(List<Category> categories)
|
||||||
|
{
|
||||||
|
throw new NotImplementedException();
|
||||||
|
}
|
||||||
|
|
||||||
|
public override Task CleanDownloads(List<object> downloads, List<Category> categoriesToClean, HashSet<string> excludedHashes)
|
||||||
|
{
|
||||||
|
throw new NotImplementedException();
|
||||||
|
}
|
||||||
|
|
||||||
|
public override Task DeleteDownload(string hash)
|
||||||
{
|
{
|
||||||
throw new NotImplementedException();
|
throw new NotImplementedException();
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -1,10 +1,12 @@
|
|||||||
using System.Collections.Concurrent;
|
using System.Collections.Concurrent;
|
||||||
using System.Text.RegularExpressions;
|
using System.Text.RegularExpressions;
|
||||||
using Common.Configuration.ContentBlocker;
|
using Common.Configuration.ContentBlocker;
|
||||||
|
using Common.Configuration.DownloadCleaner;
|
||||||
|
using Infrastructure.Interceptors;
|
||||||
|
|
||||||
namespace Infrastructure.Verticals.DownloadClient;
|
namespace Infrastructure.Verticals.DownloadClient;
|
||||||
|
|
||||||
public interface IDownloadService : IDisposable
|
public interface IDownloadService : IDisposable, IDryRunService
|
||||||
{
|
{
|
||||||
public Task LoginAsync();
|
public Task LoginAsync();
|
||||||
|
|
||||||
@@ -29,8 +31,23 @@ public interface IDownloadService : IDisposable
|
|||||||
ConcurrentBag<Regex> regexes
|
ConcurrentBag<Regex> regexes
|
||||||
);
|
);
|
||||||
|
|
||||||
|
/// <summary>
|
||||||
|
/// Fetches all downloads.
|
||||||
|
/// </summary>
|
||||||
|
/// <param name="categories">The categories by which to filter the downloads.</param>
|
||||||
|
/// <returns>A list of downloads for the provided categories.</returns>
|
||||||
|
Task<List<object>?> GetAllDownloadsToBeCleaned(List<Category> categories);
|
||||||
|
|
||||||
|
/// <summary>
|
||||||
|
/// Cleans the downloads.
|
||||||
|
/// </summary>
|
||||||
|
/// <param name="downloads"></param>
|
||||||
|
/// <param name="categoriesToClean">The categories that should be cleaned.</param>
|
||||||
|
/// <param name="excludedHashes">The hashes that should not be cleaned.</param>
|
||||||
|
public abstract Task CleanDownloads(List<object> downloads, List<Category> categoriesToClean, HashSet<string> excludedHashes);
|
||||||
|
|
||||||
/// <summary>
|
/// <summary>
|
||||||
/// Deletes a download item.
|
/// Deletes a download item.
|
||||||
/// </summary>
|
/// </summary>
|
||||||
public Task Delete(string hash);
|
public Task DeleteDownload(string hash);
|
||||||
}
|
}
|
||||||
@@ -0,0 +1,5 @@
|
|||||||
|
namespace Infrastructure.Verticals.DownloadClient.QBittorrent;
|
||||||
|
|
||||||
|
public interface IQBitService : IDownloadService
|
||||||
|
{
|
||||||
|
}
|
||||||
@@ -1,34 +1,46 @@
|
|||||||
using System.Collections.Concurrent;
|
using System.Collections.Concurrent;
|
||||||
using System.Text.RegularExpressions;
|
using System.Text.RegularExpressions;
|
||||||
|
using Common.Attributes;
|
||||||
using Common.Configuration.ContentBlocker;
|
using Common.Configuration.ContentBlocker;
|
||||||
|
using Common.Configuration.DownloadCleaner;
|
||||||
using Common.Configuration.DownloadClient;
|
using Common.Configuration.DownloadClient;
|
||||||
using Common.Configuration.QueueCleaner;
|
using Common.Configuration.QueueCleaner;
|
||||||
using Common.Helpers;
|
using Common.Helpers;
|
||||||
using Domain.Enums;
|
using Domain.Enums;
|
||||||
using Infrastructure.Verticals.ContentBlocker;
|
using Infrastructure.Verticals.ContentBlocker;
|
||||||
|
using Infrastructure.Verticals.Context;
|
||||||
using Infrastructure.Verticals.ItemStriker;
|
using Infrastructure.Verticals.ItemStriker;
|
||||||
|
using Infrastructure.Verticals.Notifications;
|
||||||
using Microsoft.Extensions.Caching.Memory;
|
using Microsoft.Extensions.Caching.Memory;
|
||||||
using Microsoft.Extensions.Logging;
|
using Microsoft.Extensions.Logging;
|
||||||
using Microsoft.Extensions.Options;
|
using Microsoft.Extensions.Options;
|
||||||
using QBittorrent.Client;
|
using QBittorrent.Client;
|
||||||
|
using Category = Common.Configuration.DownloadCleaner.Category;
|
||||||
|
|
||||||
namespace Infrastructure.Verticals.DownloadClient.QBittorrent;
|
namespace Infrastructure.Verticals.DownloadClient.QBittorrent;
|
||||||
|
|
||||||
public sealed class QBitService : DownloadServiceBase
|
public class QBitService : DownloadService, IQBitService
|
||||||
{
|
{
|
||||||
private readonly QBitConfig _config;
|
private readonly QBitConfig _config;
|
||||||
private readonly QBittorrentClient _client;
|
private readonly QBittorrentClient _client;
|
||||||
|
|
||||||
|
/// <inheritdoc/>
|
||||||
|
public QBitService()
|
||||||
|
{
|
||||||
|
}
|
||||||
|
|
||||||
public QBitService(
|
public QBitService(
|
||||||
ILogger<QBitService> logger,
|
ILogger<QBitService> logger,
|
||||||
IHttpClientFactory httpClientFactory,
|
IHttpClientFactory httpClientFactory,
|
||||||
IOptions<QBitConfig> config,
|
IOptions<QBitConfig> config,
|
||||||
IOptions<QueueCleanerConfig> queueCleanerConfig,
|
IOptions<QueueCleanerConfig> queueCleanerConfig,
|
||||||
IOptions<ContentBlockerConfig> contentBlockerConfig,
|
IOptions<ContentBlockerConfig> contentBlockerConfig,
|
||||||
|
IOptions<DownloadCleanerConfig> downloadCleanerConfig,
|
||||||
IMemoryCache cache,
|
IMemoryCache cache,
|
||||||
FilenameEvaluator filenameEvaluator,
|
IFilenameEvaluator filenameEvaluator,
|
||||||
Striker striker
|
IStriker striker,
|
||||||
) : base(logger, queueCleanerConfig, contentBlockerConfig, cache, filenameEvaluator, striker)
|
NotificationPublisher notifier
|
||||||
|
) : base(logger, queueCleanerConfig, contentBlockerConfig, downloadCleanerConfig, cache, filenameEvaluator, striker, notifier)
|
||||||
{
|
{
|
||||||
_config = config.Value;
|
_config = config.Value;
|
||||||
_config.Validate();
|
_config.Validate();
|
||||||
@@ -188,18 +200,99 @@ public sealed class QBitService : DownloadServiceBase
|
|||||||
|
|
||||||
foreach (int fileIndex in unwantedFiles)
|
foreach (int fileIndex in unwantedFiles)
|
||||||
{
|
{
|
||||||
await _client.SetFilePriorityAsync(hash, fileIndex, TorrentContentPriority.Skip);
|
await ((QBitService)Proxy).SkipFile(hash, fileIndex);
|
||||||
}
|
}
|
||||||
|
|
||||||
return result;
|
return result;
|
||||||
}
|
}
|
||||||
|
|
||||||
/// <inheritdoc/>
|
/// <inheritdoc/>
|
||||||
public override async Task Delete(string hash)
|
public override async Task<List<object>?> GetAllDownloadsToBeCleaned(List<Category> categories) =>
|
||||||
|
(await _client.GetTorrentListAsync(new()
|
||||||
|
{
|
||||||
|
Filter = TorrentListFilter.Seeding
|
||||||
|
}))
|
||||||
|
?.Where(x => !string.IsNullOrEmpty(x.Hash))
|
||||||
|
.Where(x => categories.Any(cat => cat.Name.Equals(x.Category, StringComparison.InvariantCultureIgnoreCase)))
|
||||||
|
.Cast<object>()
|
||||||
|
.ToList();
|
||||||
|
|
||||||
|
/// <inheritdoc/>
|
||||||
|
public override async Task CleanDownloads(List<object> downloads, List<Category> categoriesToClean, HashSet<string> excludedHashes)
|
||||||
|
{
|
||||||
|
foreach (TorrentInfo download in downloads)
|
||||||
|
{
|
||||||
|
if (string.IsNullOrEmpty(download.Hash))
|
||||||
|
{
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
|
||||||
|
Category? category = categoriesToClean
|
||||||
|
.FirstOrDefault(x => download.Category.Equals(x.Name, StringComparison.InvariantCultureIgnoreCase));
|
||||||
|
|
||||||
|
if (category is null)
|
||||||
|
{
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
|
||||||
|
if (excludedHashes.Any(x => x.Equals(download.Hash, StringComparison.InvariantCultureIgnoreCase)))
|
||||||
|
{
|
||||||
|
_logger.LogDebug("skip | download is used by an arr | {name}", download.Name);
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
|
||||||
|
if (!_downloadCleanerConfig.DeletePrivate)
|
||||||
|
{
|
||||||
|
TorrentProperties? torrentProperties = await _client.GetTorrentPropertiesAsync(download.Hash);
|
||||||
|
|
||||||
|
bool isPrivate = torrentProperties.AdditionalData.TryGetValue("is_private", out var dictValue) &&
|
||||||
|
bool.TryParse(dictValue?.ToString(), out bool boolValue)
|
||||||
|
&& boolValue;
|
||||||
|
|
||||||
|
if (isPrivate)
|
||||||
|
{
|
||||||
|
_logger.LogDebug("skip | download is private | {name}", download.Name);
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
ContextProvider.Set("downloadName", download.Name);
|
||||||
|
ContextProvider.Set("hash", download.Hash);
|
||||||
|
|
||||||
|
SeedingCheckResult result = ShouldCleanDownload(download.Ratio, download.SeedingTime ?? TimeSpan.Zero, category);
|
||||||
|
|
||||||
|
if (!result.ShouldClean)
|
||||||
|
{
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
|
||||||
|
await ((QBitService)Proxy).DeleteDownload(download.Hash);
|
||||||
|
|
||||||
|
_logger.LogInformation(
|
||||||
|
"download cleaned | {reason} reached | {name}",
|
||||||
|
result.Reason is CleanReason.MaxRatioReached
|
||||||
|
? "MAX_RATIO & MIN_SEED_TIME"
|
||||||
|
: "MAX_SEED_TIME",
|
||||||
|
download.Name
|
||||||
|
);
|
||||||
|
|
||||||
|
await _notifier.NotifyDownloadCleaned(download.Ratio, download.SeedingTime ?? TimeSpan.Zero, category.Name, result.Reason);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// <inheritdoc/>
|
||||||
|
[DryRunSafeguard]
|
||||||
|
public override async Task DeleteDownload(string hash)
|
||||||
{
|
{
|
||||||
await _client.DeleteAsync(hash, deleteDownloadedData: true);
|
await _client.DeleteAsync(hash, deleteDownloadedData: true);
|
||||||
}
|
}
|
||||||
|
|
||||||
|
[DryRunSafeguard]
|
||||||
|
protected virtual async Task SkipFile(string hash, int fileIndex)
|
||||||
|
{
|
||||||
|
await _client.SetFilePriorityAsync(hash, fileIndex, TorrentContentPriority.Skip);
|
||||||
|
}
|
||||||
|
|
||||||
public override void Dispose()
|
public override void Dispose()
|
||||||
{
|
{
|
||||||
_client.Dispose();
|
_client.Dispose();
|
||||||
|
|||||||
@@ -0,0 +1,9 @@
|
|||||||
|
using Domain.Enums;
|
||||||
|
|
||||||
|
namespace Infrastructure.Verticals.DownloadClient;
|
||||||
|
|
||||||
|
public sealed record SeedingCheckResult
|
||||||
|
{
|
||||||
|
public bool ShouldClean { get; set; }
|
||||||
|
public CleanReason Reason { get; set; }
|
||||||
|
}
|
||||||
@@ -0,0 +1,5 @@
|
|||||||
|
namespace Infrastructure.Verticals.DownloadClient.Transmission;
|
||||||
|
|
||||||
|
public interface ITransmissionService : IDownloadService
|
||||||
|
{
|
||||||
|
}
|
||||||
@@ -1,12 +1,16 @@
|
|||||||
using System.Collections.Concurrent;
|
using System.Collections.Concurrent;
|
||||||
using System.Text.RegularExpressions;
|
using System.Text.RegularExpressions;
|
||||||
|
using Common.Attributes;
|
||||||
using Common.Configuration.ContentBlocker;
|
using Common.Configuration.ContentBlocker;
|
||||||
|
using Common.Configuration.DownloadCleaner;
|
||||||
using Common.Configuration.DownloadClient;
|
using Common.Configuration.DownloadClient;
|
||||||
using Common.Configuration.QueueCleaner;
|
using Common.Configuration.QueueCleaner;
|
||||||
using Common.Helpers;
|
using Common.Helpers;
|
||||||
using Domain.Enums;
|
using Domain.Enums;
|
||||||
using Infrastructure.Verticals.ContentBlocker;
|
using Infrastructure.Verticals.ContentBlocker;
|
||||||
|
using Infrastructure.Verticals.Context;
|
||||||
using Infrastructure.Verticals.ItemStriker;
|
using Infrastructure.Verticals.ItemStriker;
|
||||||
|
using Infrastructure.Verticals.Notifications;
|
||||||
using Microsoft.Extensions.Caching.Memory;
|
using Microsoft.Extensions.Caching.Memory;
|
||||||
using Microsoft.Extensions.Logging;
|
using Microsoft.Extensions.Logging;
|
||||||
using Microsoft.Extensions.Options;
|
using Microsoft.Extensions.Options;
|
||||||
@@ -16,22 +20,29 @@ using Transmission.API.RPC.Entity;
|
|||||||
|
|
||||||
namespace Infrastructure.Verticals.DownloadClient.Transmission;
|
namespace Infrastructure.Verticals.DownloadClient.Transmission;
|
||||||
|
|
||||||
public sealed class TransmissionService : DownloadServiceBase
|
public class TransmissionService : DownloadService, ITransmissionService
|
||||||
{
|
{
|
||||||
private readonly TransmissionConfig _config;
|
private readonly TransmissionConfig _config;
|
||||||
private readonly Client _client;
|
private readonly Client _client;
|
||||||
private TorrentInfo[]? _torrentsCache;
|
private TorrentInfo[]? _torrentsCache;
|
||||||
|
|
||||||
|
/// <inheritdoc/>
|
||||||
|
public TransmissionService()
|
||||||
|
{
|
||||||
|
}
|
||||||
|
|
||||||
public TransmissionService(
|
public TransmissionService(
|
||||||
IHttpClientFactory httpClientFactory,
|
IHttpClientFactory httpClientFactory,
|
||||||
ILogger<TransmissionService> logger,
|
ILogger<TransmissionService> logger,
|
||||||
IOptions<TransmissionConfig> config,
|
IOptions<TransmissionConfig> config,
|
||||||
IOptions<QueueCleanerConfig> queueCleanerConfig,
|
IOptions<QueueCleanerConfig> queueCleanerConfig,
|
||||||
IOptions<ContentBlockerConfig> contentBlockerConfig,
|
IOptions<ContentBlockerConfig> contentBlockerConfig,
|
||||||
|
IOptions<DownloadCleanerConfig> downloadCleanerConfig,
|
||||||
IMemoryCache cache,
|
IMemoryCache cache,
|
||||||
FilenameEvaluator filenameEvaluator,
|
IFilenameEvaluator filenameEvaluator,
|
||||||
Striker striker
|
IStriker striker,
|
||||||
) : base(logger, queueCleanerConfig, contentBlockerConfig, cache, filenameEvaluator, striker)
|
NotificationPublisher notifier
|
||||||
|
) : base(logger, queueCleanerConfig, contentBlockerConfig, downloadCleanerConfig, cache, filenameEvaluator, striker, notifier)
|
||||||
{
|
{
|
||||||
_config = config.Value;
|
_config = config.Value;
|
||||||
_config.Validate();
|
_config.Validate();
|
||||||
@@ -164,16 +175,96 @@ public sealed class TransmissionService : DownloadServiceBase
|
|||||||
|
|
||||||
_logger.LogDebug("changing priorities | torrent {hash}", hash);
|
_logger.LogDebug("changing priorities | torrent {hash}", hash);
|
||||||
|
|
||||||
await _client.TorrentSetAsync(new TorrentSettings
|
await ((TransmissionService)Proxy).SetUnwantedFiles(torrent.Id, unwantedFiles.ToArray());
|
||||||
{
|
|
||||||
Ids = [ torrent.Id ],
|
|
||||||
FilesUnwanted = unwantedFiles.ToArray(),
|
|
||||||
});
|
|
||||||
|
|
||||||
return result;
|
return result;
|
||||||
}
|
}
|
||||||
|
|
||||||
public override async Task Delete(string hash)
|
/// <inheritdoc/>
|
||||||
|
public override async Task<List<object>?> GetAllDownloadsToBeCleaned(List<Category> categories)
|
||||||
|
{
|
||||||
|
string[] fields = [
|
||||||
|
TorrentFields.FILES,
|
||||||
|
TorrentFields.FILE_STATS,
|
||||||
|
TorrentFields.HASH_STRING,
|
||||||
|
TorrentFields.ID,
|
||||||
|
TorrentFields.ETA,
|
||||||
|
TorrentFields.NAME,
|
||||||
|
TorrentFields.STATUS,
|
||||||
|
TorrentFields.IS_PRIVATE,
|
||||||
|
TorrentFields.DOWNLOADED_EVER,
|
||||||
|
TorrentFields.DOWNLOAD_DIR,
|
||||||
|
TorrentFields.SECONDS_SEEDING,
|
||||||
|
TorrentFields.UPLOAD_RATIO
|
||||||
|
];
|
||||||
|
|
||||||
|
return (await _client.TorrentGetAsync(fields))
|
||||||
|
?.Torrents
|
||||||
|
?.Where(x => !string.IsNullOrEmpty(x.HashString))
|
||||||
|
.Where(x => x.Status is 5 or 6)
|
||||||
|
.Where(x => categories
|
||||||
|
.Any(cat => x.DownloadDir?.EndsWith(cat.Name, StringComparison.InvariantCultureIgnoreCase) is true)
|
||||||
|
)
|
||||||
|
.Cast<object>()
|
||||||
|
.ToList();
|
||||||
|
}
|
||||||
|
|
||||||
|
/// <inheritdoc/>
|
||||||
|
public override async Task CleanDownloads(List<object> downloads, List<Category> categoriesToClean, HashSet<string> excludedHashes)
|
||||||
|
{
|
||||||
|
foreach (TorrentInfo download in downloads)
|
||||||
|
{
|
||||||
|
if (string.IsNullOrEmpty(download.HashString))
|
||||||
|
{
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
|
||||||
|
Category? category = categoriesToClean
|
||||||
|
.FirstOrDefault(x => download.DownloadDir?.EndsWith(x.Name, StringComparison.InvariantCultureIgnoreCase) is true);
|
||||||
|
|
||||||
|
if (category is null)
|
||||||
|
{
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
|
||||||
|
if (excludedHashes.Any(x => x.Equals(download.HashString, StringComparison.InvariantCultureIgnoreCase)))
|
||||||
|
{
|
||||||
|
_logger.LogDebug("skip | download is used by an arr | {name}", download.Name);
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
|
||||||
|
if (!_downloadCleanerConfig.DeletePrivate && download.IsPrivate is true)
|
||||||
|
{
|
||||||
|
_logger.LogDebug("skip | download is private | {name}", download.Name);
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
|
||||||
|
ContextProvider.Set("downloadName", download.Name);
|
||||||
|
ContextProvider.Set("hash", download.HashString);
|
||||||
|
|
||||||
|
TimeSpan seedingTime = TimeSpan.FromSeconds(download.SecondsSeeding ?? 0);
|
||||||
|
SeedingCheckResult result = ShouldCleanDownload(download.uploadRatio ?? 0, seedingTime, category);
|
||||||
|
|
||||||
|
if (!result.ShouldClean)
|
||||||
|
{
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
|
||||||
|
await ((TransmissionService)Proxy).RemoveDownloadAsync(download.Id);
|
||||||
|
|
||||||
|
_logger.LogInformation(
|
||||||
|
"download cleaned | {reason} reached | {name}",
|
||||||
|
result.Reason is CleanReason.MaxRatioReached
|
||||||
|
? "MAX_RATIO & MIN_SEED_TIME"
|
||||||
|
: "MAX_SEED_TIME",
|
||||||
|
download.Name
|
||||||
|
);
|
||||||
|
|
||||||
|
await _notifier.NotifyDownloadCleaned(download.uploadRatio ?? 0, seedingTime, category.Name, result.Reason);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
public override async Task DeleteDownload(string hash)
|
||||||
{
|
{
|
||||||
TorrentInfo? torrent = await GetTorrentAsync(hash);
|
TorrentInfo? torrent = await GetTorrentAsync(hash);
|
||||||
|
|
||||||
@@ -189,6 +280,22 @@ public sealed class TransmissionService : DownloadServiceBase
|
|||||||
{
|
{
|
||||||
}
|
}
|
||||||
|
|
||||||
|
[DryRunSafeguard]
|
||||||
|
protected virtual async Task RemoveDownloadAsync(long downloadId)
|
||||||
|
{
|
||||||
|
await _client.TorrentRemoveAsync([downloadId], true);
|
||||||
|
}
|
||||||
|
|
||||||
|
[DryRunSafeguard]
|
||||||
|
protected virtual async Task SetUnwantedFiles(long downloadId, long[] unwantedFiles)
|
||||||
|
{
|
||||||
|
await _client.TorrentSetAsync(new TorrentSettings
|
||||||
|
{
|
||||||
|
Ids = [downloadId],
|
||||||
|
FilesUnwanted = unwantedFiles,
|
||||||
|
});
|
||||||
|
}
|
||||||
|
|
||||||
private async Task<bool> IsItemStuckAndShouldRemove(TorrentInfo torrent)
|
private async Task<bool> IsItemStuckAndShouldRemove(TorrentInfo torrent)
|
||||||
{
|
{
|
||||||
if (_queueCleanerConfig.StalledMaxStrikes is 0)
|
if (_queueCleanerConfig.StalledMaxStrikes is 0)
|
||||||
|
|||||||
@@ -0,0 +1,8 @@
|
|||||||
|
using Domain.Enums;
|
||||||
|
|
||||||
|
namespace Infrastructure.Verticals.ItemStriker;
|
||||||
|
|
||||||
|
public interface IStriker
|
||||||
|
{
|
||||||
|
Task<bool> StrikeAndCheckLimit(string hash, string itemName, ushort maxStrikes, StrikeType strikeType);
|
||||||
|
}
|
||||||
@@ -8,7 +8,7 @@ using Microsoft.Extensions.Logging;
|
|||||||
|
|
||||||
namespace Infrastructure.Verticals.ItemStriker;
|
namespace Infrastructure.Verticals.ItemStriker;
|
||||||
|
|
||||||
public class Striker
|
public sealed class Striker : IStriker
|
||||||
{
|
{
|
||||||
private readonly ILogger<Striker> _logger;
|
private readonly ILogger<Striker> _logger;
|
||||||
private readonly IMemoryCache _cache;
|
private readonly IMemoryCache _cache;
|
||||||
|
|||||||
@@ -4,6 +4,7 @@ using Domain.Enums;
|
|||||||
using Domain.Models.Arr;
|
using Domain.Models.Arr;
|
||||||
using Domain.Models.Arr.Queue;
|
using Domain.Models.Arr.Queue;
|
||||||
using Infrastructure.Verticals.Arr;
|
using Infrastructure.Verticals.Arr;
|
||||||
|
using Infrastructure.Verticals.Arr.Interfaces;
|
||||||
using Infrastructure.Verticals.DownloadClient;
|
using Infrastructure.Verticals.DownloadClient;
|
||||||
using Infrastructure.Verticals.Notifications;
|
using Infrastructure.Verticals.Notifications;
|
||||||
using Microsoft.Extensions.Logging;
|
using Microsoft.Extensions.Logging;
|
||||||
@@ -11,16 +12,16 @@ using Microsoft.Extensions.Options;
|
|||||||
|
|
||||||
namespace Infrastructure.Verticals.Jobs;
|
namespace Infrastructure.Verticals.Jobs;
|
||||||
|
|
||||||
public abstract class GenericHandler : IDisposable
|
public abstract class GenericHandler : IHandler, IDisposable
|
||||||
{
|
{
|
||||||
protected readonly ILogger<GenericHandler> _logger;
|
protected readonly ILogger<GenericHandler> _logger;
|
||||||
protected readonly DownloadClientConfig _downloadClientConfig;
|
protected readonly DownloadClientConfig _downloadClientConfig;
|
||||||
protected readonly SonarrConfig _sonarrConfig;
|
protected readonly SonarrConfig _sonarrConfig;
|
||||||
protected readonly RadarrConfig _radarrConfig;
|
protected readonly RadarrConfig _radarrConfig;
|
||||||
protected readonly LidarrConfig _lidarrConfig;
|
protected readonly LidarrConfig _lidarrConfig;
|
||||||
protected readonly SonarrClient _sonarrClient;
|
protected readonly ISonarrClient _sonarrClient;
|
||||||
protected readonly RadarrClient _radarrClient;
|
protected readonly IRadarrClient _radarrClient;
|
||||||
protected readonly LidarrClient _lidarrClient;
|
protected readonly ILidarrClient _lidarrClient;
|
||||||
protected readonly ArrQueueIterator _arrArrQueueIterator;
|
protected readonly ArrQueueIterator _arrArrQueueIterator;
|
||||||
protected readonly IDownloadService _downloadService;
|
protected readonly IDownloadService _downloadService;
|
||||||
protected readonly NotificationPublisher _notifier;
|
protected readonly NotificationPublisher _notifier;
|
||||||
@@ -31,9 +32,9 @@ public abstract class GenericHandler : IDisposable
|
|||||||
IOptions<SonarrConfig> sonarrConfig,
|
IOptions<SonarrConfig> sonarrConfig,
|
||||||
IOptions<RadarrConfig> radarrConfig,
|
IOptions<RadarrConfig> radarrConfig,
|
||||||
IOptions<LidarrConfig> lidarrConfig,
|
IOptions<LidarrConfig> lidarrConfig,
|
||||||
SonarrClient sonarrClient,
|
ISonarrClient sonarrClient,
|
||||||
RadarrClient radarrClient,
|
IRadarrClient radarrClient,
|
||||||
LidarrClient lidarrClient,
|
ILidarrClient lidarrClient,
|
||||||
ArrQueueIterator arrArrQueueIterator,
|
ArrQueueIterator arrArrQueueIterator,
|
||||||
DownloadServiceFactory downloadServiceFactory,
|
DownloadServiceFactory downloadServiceFactory,
|
||||||
NotificationPublisher notifier
|
NotificationPublisher notifier
|
||||||
@@ -68,7 +69,7 @@ public abstract class GenericHandler : IDisposable
|
|||||||
|
|
||||||
protected abstract Task ProcessInstanceAsync(ArrInstance instance, InstanceType instanceType);
|
protected abstract Task ProcessInstanceAsync(ArrInstance instance, InstanceType instanceType);
|
||||||
|
|
||||||
private async Task ProcessArrConfigAsync(ArrConfig config, InstanceType instanceType)
|
protected async Task ProcessArrConfigAsync(ArrConfig config, InstanceType instanceType, bool throwOnFailure = false)
|
||||||
{
|
{
|
||||||
if (!config.Enabled)
|
if (!config.Enabled)
|
||||||
{
|
{
|
||||||
@@ -84,11 +85,16 @@ public abstract class GenericHandler : IDisposable
|
|||||||
catch (Exception exception)
|
catch (Exception exception)
|
||||||
{
|
{
|
||||||
_logger.LogError(exception, "failed to clean {type} instance | {url}", instanceType, arrInstance.Url);
|
_logger.LogError(exception, "failed to clean {type} instance | {url}", instanceType, arrInstance.Url);
|
||||||
|
|
||||||
|
if (throwOnFailure)
|
||||||
|
{
|
||||||
|
throw;
|
||||||
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
protected ArrClient GetClient(InstanceType type) =>
|
protected IArrClient GetClient(InstanceType type) =>
|
||||||
type switch
|
type switch
|
||||||
{
|
{
|
||||||
InstanceType.Sonarr => _sonarrClient,
|
InstanceType.Sonarr => _sonarrClient,
|
||||||
|
|||||||
@@ -0,0 +1,6 @@
|
|||||||
|
namespace Infrastructure.Verticals.Jobs;
|
||||||
|
|
||||||
|
public interface IHandler
|
||||||
|
{
|
||||||
|
Task ExecuteAsync();
|
||||||
|
}
|
||||||
@@ -27,9 +27,12 @@ public sealed class NotificationConsumer<T> : IConsumer<T> where T : Notificatio
|
|||||||
case StalledStrikeNotification stalledMessage:
|
case StalledStrikeNotification stalledMessage:
|
||||||
await _notificationService.Notify(stalledMessage);
|
await _notificationService.Notify(stalledMessage);
|
||||||
break;
|
break;
|
||||||
case QueueItemDeleteNotification queueItemDeleteMessage:
|
case QueueItemDeletedNotification queueItemDeleteMessage:
|
||||||
await _notificationService.Notify(queueItemDeleteMessage);
|
await _notificationService.Notify(queueItemDeleteMessage);
|
||||||
break;
|
break;
|
||||||
|
case DownloadCleanedNotification downloadCleanedNotification:
|
||||||
|
await _notificationService.Notify(downloadCleanedNotification);
|
||||||
|
break;
|
||||||
default:
|
default:
|
||||||
throw new NotImplementedException();
|
throw new NotImplementedException();
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -6,5 +6,7 @@ public interface INotificationFactory
|
|||||||
|
|
||||||
List<INotificationProvider> OnStalledStrikeEnabled();
|
List<INotificationProvider> OnStalledStrikeEnabled();
|
||||||
|
|
||||||
List<INotificationProvider> OnQueueItemDeleteEnabled();
|
List<INotificationProvider> OnQueueItemDeletedEnabled();
|
||||||
|
|
||||||
|
List<INotificationProvider> OnDownloadCleanedEnabled();
|
||||||
}
|
}
|
||||||
@@ -13,5 +13,7 @@ public interface INotificationProvider
|
|||||||
|
|
||||||
Task OnStalledStrike(StalledStrikeNotification notification);
|
Task OnStalledStrike(StalledStrikeNotification notification);
|
||||||
|
|
||||||
Task OnQueueItemDelete(QueueItemDeleteNotification notification);
|
Task OnQueueItemDeleted(QueueItemDeletedNotification notification);
|
||||||
|
|
||||||
|
Task OnDownloadCleaned(DownloadCleanedNotification notification);
|
||||||
}
|
}
|
||||||
@@ -0,0 +1,14 @@
|
|||||||
|
using Domain.Enums;
|
||||||
|
|
||||||
|
namespace Infrastructure.Verticals.Notifications.Models;
|
||||||
|
|
||||||
|
public record ArrNotification : Notification
|
||||||
|
{
|
||||||
|
public required InstanceType InstanceType { get; init; }
|
||||||
|
|
||||||
|
public required Uri InstanceUrl { get; init; }
|
||||||
|
|
||||||
|
public required string Hash { get; init; }
|
||||||
|
|
||||||
|
public Uri? Image { get; init; }
|
||||||
|
}
|
||||||
+1
-1
@@ -1,5 +1,5 @@
|
|||||||
namespace Infrastructure.Verticals.Notifications.Models;
|
namespace Infrastructure.Verticals.Notifications.Models;
|
||||||
|
|
||||||
public sealed record QueueItemDeleteNotification : Notification
|
public sealed record DownloadCleanedNotification : Notification
|
||||||
{
|
{
|
||||||
}
|
}
|
||||||
+1
-1
@@ -1,5 +1,5 @@
|
|||||||
namespace Infrastructure.Verticals.Notifications.Models;
|
namespace Infrastructure.Verticals.Notifications.Models;
|
||||||
|
|
||||||
public sealed record FailedImportStrikeNotification : Notification
|
public sealed record FailedImportStrikeNotification : ArrNotification
|
||||||
{
|
{
|
||||||
}
|
}
|
||||||
@@ -1,20 +1,12 @@
|
|||||||
using Domain.Enums;
|
namespace Infrastructure.Verticals.Notifications.Models;
|
||||||
|
|
||||||
namespace Infrastructure.Verticals.Notifications.Models;
|
public abstract record Notification
|
||||||
|
|
||||||
public record Notification
|
|
||||||
{
|
{
|
||||||
public required InstanceType InstanceType { get; init; }
|
|
||||||
|
|
||||||
public required Uri InstanceUrl { get; init; }
|
|
||||||
|
|
||||||
public required string Hash { get; init; }
|
|
||||||
|
|
||||||
public required string Title { get; init; }
|
public required string Title { get; init; }
|
||||||
|
|
||||||
public required string Description { get; init; }
|
public required string Description { get; init; }
|
||||||
|
|
||||||
public Uri? Image { get; init; }
|
|
||||||
|
|
||||||
public List<NotificationField>? Fields { get; init; }
|
public List<NotificationField>? Fields { get; init; }
|
||||||
|
|
||||||
|
public NotificationLevel Level { get; init; }
|
||||||
}
|
}
|
||||||
@@ -0,0 +1,9 @@
|
|||||||
|
namespace Infrastructure.Verticals.Notifications.Models;
|
||||||
|
|
||||||
|
public enum NotificationLevel
|
||||||
|
{
|
||||||
|
Test,
|
||||||
|
Information,
|
||||||
|
Warning,
|
||||||
|
Important
|
||||||
|
}
|
||||||
@@ -0,0 +1,5 @@
|
|||||||
|
namespace Infrastructure.Verticals.Notifications.Models;
|
||||||
|
|
||||||
|
public sealed record QueueItemDeletedNotification : ArrNotification
|
||||||
|
{
|
||||||
|
}
|
||||||
@@ -1,5 +1,5 @@
|
|||||||
namespace Infrastructure.Verticals.Notifications.Models;
|
namespace Infrastructure.Verticals.Notifications.Models;
|
||||||
|
|
||||||
public sealed record StalledStrikeNotification : Notification
|
public sealed record StalledStrikeNotification : ArrNotification
|
||||||
{
|
{
|
||||||
}
|
}
|
||||||
@@ -1,4 +1,3 @@
|
|||||||
using Domain.Enums;
|
|
||||||
using Infrastructure.Verticals.Notifications.Models;
|
using Infrastructure.Verticals.Notifications.Models;
|
||||||
using Mapster;
|
using Mapster;
|
||||||
using Microsoft.Extensions.Options;
|
using Microsoft.Extensions.Options;
|
||||||
@@ -12,6 +11,7 @@ public class NotifiarrProvider : NotificationProvider
|
|||||||
|
|
||||||
private const string WarningColor = "f0ad4e";
|
private const string WarningColor = "f0ad4e";
|
||||||
private const string ImportantColor = "bb2124";
|
private const string ImportantColor = "bb2124";
|
||||||
|
private const string Logo = "https://github.com/flmorg/cleanuperr/blob/main/Logo/48.png?raw=true";
|
||||||
|
|
||||||
public NotifiarrProvider(IOptions<NotifiarrConfig> config, INotifiarrProxy proxy)
|
public NotifiarrProvider(IOptions<NotifiarrConfig> config, INotifiarrProxy proxy)
|
||||||
: base(config)
|
: base(config)
|
||||||
@@ -32,12 +32,17 @@ public class NotifiarrProvider : NotificationProvider
|
|||||||
await _proxy.SendNotification(BuildPayload(notification, WarningColor), _config);
|
await _proxy.SendNotification(BuildPayload(notification, WarningColor), _config);
|
||||||
}
|
}
|
||||||
|
|
||||||
public override async Task OnQueueItemDelete(QueueItemDeleteNotification notification)
|
public override async Task OnQueueItemDeleted(QueueItemDeletedNotification notification)
|
||||||
{
|
{
|
||||||
await _proxy.SendNotification(BuildPayload(notification, ImportantColor), _config);
|
await _proxy.SendNotification(BuildPayload(notification, ImportantColor), _config);
|
||||||
}
|
}
|
||||||
|
|
||||||
private NotifiarrPayload BuildPayload(Notification notification, string color)
|
public override async Task OnDownloadCleaned(DownloadCleanedNotification notification)
|
||||||
|
{
|
||||||
|
await _proxy.SendNotification(BuildPayload(notification), _config);
|
||||||
|
}
|
||||||
|
|
||||||
|
private NotifiarrPayload BuildPayload(ArrNotification notification, string color)
|
||||||
{
|
{
|
||||||
NotifiarrPayload payload = new()
|
NotifiarrPayload payload = new()
|
||||||
{
|
{
|
||||||
@@ -47,7 +52,7 @@ public class NotifiarrProvider : NotificationProvider
|
|||||||
Text = new()
|
Text = new()
|
||||||
{
|
{
|
||||||
Title = notification.Title,
|
Title = notification.Title,
|
||||||
Icon = "https://github.com/flmorg/cleanuperr/blob/main/Logo/48.png?raw=true",
|
Icon = Logo,
|
||||||
Description = notification.Description,
|
Description = notification.Description,
|
||||||
Fields = new()
|
Fields = new()
|
||||||
{
|
{
|
||||||
@@ -62,7 +67,7 @@ public class NotifiarrProvider : NotificationProvider
|
|||||||
},
|
},
|
||||||
Images = new()
|
Images = new()
|
||||||
{
|
{
|
||||||
Thumbnail = new Uri("https://github.com/flmorg/cleanuperr/blob/main/Logo/48.png?raw=true"),
|
Thumbnail = new Uri(Logo),
|
||||||
Image = notification.Image
|
Image = notification.Image
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
@@ -72,4 +77,32 @@ public class NotifiarrProvider : NotificationProvider
|
|||||||
|
|
||||||
return payload;
|
return payload;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
private NotifiarrPayload BuildPayload(DownloadCleanedNotification notification)
|
||||||
|
{
|
||||||
|
NotifiarrPayload payload = new()
|
||||||
|
{
|
||||||
|
Discord = new()
|
||||||
|
{
|
||||||
|
Color = ImportantColor,
|
||||||
|
Text = new()
|
||||||
|
{
|
||||||
|
Title = notification.Title,
|
||||||
|
Icon = Logo,
|
||||||
|
Description = notification.Description,
|
||||||
|
Fields = notification.Fields?.Adapt<List<Field>>() ?? []
|
||||||
|
},
|
||||||
|
Ids = new Ids
|
||||||
|
{
|
||||||
|
Channel = _config.ChannelId
|
||||||
|
},
|
||||||
|
Images = new()
|
||||||
|
{
|
||||||
|
Thumbnail = new Uri(Logo)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
};
|
||||||
|
|
||||||
|
return payload;
|
||||||
|
}
|
||||||
}
|
}
|
||||||
@@ -25,8 +25,13 @@ public class NotificationFactory : INotificationFactory
|
|||||||
.Where(n => n.Config.OnStalledStrike)
|
.Where(n => n.Config.OnStalledStrike)
|
||||||
.ToList();
|
.ToList();
|
||||||
|
|
||||||
public List<INotificationProvider> OnQueueItemDeleteEnabled() =>
|
public List<INotificationProvider> OnQueueItemDeletedEnabled() =>
|
||||||
ActiveProviders()
|
ActiveProviders()
|
||||||
.Where(n => n.Config.OnQueueItemDelete)
|
.Where(n => n.Config.OnQueueItemDeleted)
|
||||||
|
.ToList();
|
||||||
|
|
||||||
|
public List<INotificationProvider> OnDownloadCleanedEnabled() =>
|
||||||
|
ActiveProviders()
|
||||||
|
.Where(n => n.Config.OnDownloadCleaned)
|
||||||
.ToList();
|
.ToList();
|
||||||
}
|
}
|
||||||
@@ -19,5 +19,7 @@ public abstract class NotificationProvider : INotificationProvider
|
|||||||
|
|
||||||
public abstract Task OnStalledStrike(StalledStrikeNotification notification);
|
public abstract Task OnStalledStrike(StalledStrikeNotification notification);
|
||||||
|
|
||||||
public abstract Task OnQueueItemDelete(QueueItemDeleteNotification notification);
|
public abstract Task OnQueueItemDeleted(QueueItemDeletedNotification notification);
|
||||||
|
|
||||||
|
public abstract Task OnDownloadCleaned(DownloadCleanedNotification notification);
|
||||||
}
|
}
|
||||||
@@ -1,6 +1,9 @@
|
|||||||
using Common.Configuration.Arr;
|
using System.Globalization;
|
||||||
|
using Common.Attributes;
|
||||||
|
using Common.Configuration.Arr;
|
||||||
using Domain.Enums;
|
using Domain.Enums;
|
||||||
using Domain.Models.Arr.Queue;
|
using Domain.Models.Arr.Queue;
|
||||||
|
using Infrastructure.Interceptors;
|
||||||
using Infrastructure.Verticals.Context;
|
using Infrastructure.Verticals.Context;
|
||||||
using Infrastructure.Verticals.Notifications.Models;
|
using Infrastructure.Verticals.Notifications.Models;
|
||||||
using Mapster;
|
using Mapster;
|
||||||
@@ -9,27 +12,35 @@ using Microsoft.Extensions.Logging;
|
|||||||
|
|
||||||
namespace Infrastructure.Verticals.Notifications;
|
namespace Infrastructure.Verticals.Notifications;
|
||||||
|
|
||||||
public sealed class NotificationPublisher
|
public class NotificationPublisher : InterceptedService, IDryRunService
|
||||||
{
|
{
|
||||||
private readonly ILogger<NotificationPublisher> _logger;
|
private readonly ILogger<NotificationPublisher> _logger;
|
||||||
private readonly IBus _messageBus;
|
private readonly IBus _messageBus;
|
||||||
|
|
||||||
|
/// <summary>
|
||||||
|
/// Constructor to be used by interceptors.
|
||||||
|
/// </summary>
|
||||||
|
public NotificationPublisher()
|
||||||
|
{
|
||||||
|
}
|
||||||
|
|
||||||
public NotificationPublisher(ILogger<NotificationPublisher> logger, IBus messageBus)
|
public NotificationPublisher(ILogger<NotificationPublisher> logger, IBus messageBus)
|
||||||
{
|
{
|
||||||
_logger = logger;
|
_logger = logger;
|
||||||
_messageBus = messageBus;
|
_messageBus = messageBus;
|
||||||
}
|
}
|
||||||
|
|
||||||
public async Task NotifyStrike(StrikeType strikeType, int strikeCount)
|
[DryRunSafeguard]
|
||||||
|
public virtual async Task NotifyStrike(StrikeType strikeType, int strikeCount)
|
||||||
{
|
{
|
||||||
try
|
try
|
||||||
{
|
{
|
||||||
QueueRecord record = GetRecordFromContext();
|
QueueRecord record = ContextProvider.Get<QueueRecord>(nameof(QueueRecord));
|
||||||
InstanceType instanceType = GetInstanceTypeFromContext();
|
InstanceType instanceType = (InstanceType)ContextProvider.Get<object>(nameof(InstanceType));
|
||||||
Uri instanceUrl = GetInstanceUrlFromContext();
|
Uri instanceUrl = ContextProvider.Get<Uri>(nameof(ArrInstance) + nameof(ArrInstance.Url));
|
||||||
Uri? imageUrl = GetImageFromContext(record, instanceType);
|
Uri imageUrl = GetImageFromContext(record, instanceType);
|
||||||
|
|
||||||
Notification notification = new()
|
ArrNotification notification = new()
|
||||||
{
|
{
|
||||||
InstanceType = instanceType,
|
InstanceType = instanceType,
|
||||||
InstanceUrl = instanceUrl,
|
InstanceUrl = instanceUrl,
|
||||||
@@ -56,14 +67,15 @@ public sealed class NotificationPublisher
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
public async Task NotifyQueueItemDelete(bool removeFromClient, DeleteReason reason)
|
[DryRunSafeguard]
|
||||||
|
public virtual async Task NotifyQueueItemDeleted(bool removeFromClient, DeleteReason reason)
|
||||||
{
|
{
|
||||||
QueueRecord record = GetRecordFromContext();
|
QueueRecord record = ContextProvider.Get<QueueRecord>(nameof(QueueRecord));
|
||||||
InstanceType instanceType = GetInstanceTypeFromContext();
|
InstanceType instanceType = (InstanceType)ContextProvider.Get<object>(nameof(InstanceType));
|
||||||
Uri instanceUrl = GetInstanceUrlFromContext();
|
Uri instanceUrl = ContextProvider.Get<Uri>(nameof(ArrInstance) + nameof(ArrInstance.Url));
|
||||||
Uri? imageUrl = GetImageFromContext(record, instanceType);
|
Uri imageUrl = GetImageFromContext(record, instanceType);
|
||||||
|
|
||||||
Notification notification = new()
|
QueueItemDeletedNotification notification = new()
|
||||||
{
|
{
|
||||||
InstanceType = instanceType,
|
InstanceType = instanceType,
|
||||||
InstanceUrl = instanceUrl,
|
InstanceUrl = instanceUrl,
|
||||||
@@ -74,19 +86,28 @@ public sealed class NotificationPublisher
|
|||||||
Fields = [new() { Title = "Removed from download client?", Text = removeFromClient ? "Yes" : "No" }]
|
Fields = [new() { Title = "Removed from download client?", Text = removeFromClient ? "Yes" : "No" }]
|
||||||
};
|
};
|
||||||
|
|
||||||
await _messageBus.Publish(notification.Adapt<QueueItemDeleteNotification>());
|
await _messageBus.Publish(notification);
|
||||||
}
|
}
|
||||||
|
|
||||||
private static QueueRecord GetRecordFromContext() =>
|
[DryRunSafeguard]
|
||||||
ContextProvider.Get<QueueRecord>(nameof(QueueRecord)) ?? throw new Exception("failed to get record from context");
|
public virtual async Task NotifyDownloadCleaned(double ratio, TimeSpan seedingTime, string categoryName, CleanReason reason)
|
||||||
|
{
|
||||||
|
DownloadCleanedNotification notification = new()
|
||||||
|
{
|
||||||
|
Title = $"Cleaned item from download client with reason: {reason}",
|
||||||
|
Description = ContextProvider.Get<string>("downloadName"),
|
||||||
|
Fields =
|
||||||
|
[
|
||||||
|
new() { Title = "Hash", Text = ContextProvider.Get<string>("hash").ToLowerInvariant() },
|
||||||
|
new() { Title = "Category", Text = categoryName.ToLowerInvariant() },
|
||||||
|
new() { Title = "Ratio", Text = $"{ratio.ToString(CultureInfo.InvariantCulture)}%" },
|
||||||
|
new() { Title = "Seeding hours", Text = $"{Math.Round(seedingTime.TotalHours, 0).ToString(CultureInfo.InvariantCulture)}h" }
|
||||||
|
],
|
||||||
|
Level = NotificationLevel.Important
|
||||||
|
};
|
||||||
|
|
||||||
private static InstanceType GetInstanceTypeFromContext() =>
|
await _messageBus.Publish(notification);
|
||||||
(InstanceType)(ContextProvider.Get<object>(nameof(InstanceType)) ??
|
}
|
||||||
throw new Exception("failed to get instance type from context"));
|
|
||||||
|
|
||||||
private static Uri GetInstanceUrlFromContext() =>
|
|
||||||
ContextProvider.Get<Uri>(nameof(ArrInstance) + nameof(ArrInstance.Url)) ??
|
|
||||||
throw new Exception("failed to get instance url from context");
|
|
||||||
|
|
||||||
private static Uri GetImageFromContext(QueueRecord record, InstanceType instanceType) =>
|
private static Uri GetImageFromContext(QueueRecord record, InstanceType instanceType) =>
|
||||||
instanceType switch
|
instanceType switch
|
||||||
|
|||||||
@@ -44,13 +44,28 @@ public class NotificationService
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
public async Task Notify(QueueItemDeleteNotification notification)
|
public async Task Notify(QueueItemDeletedNotification notification)
|
||||||
{
|
{
|
||||||
foreach (INotificationProvider provider in _notificationFactory.OnQueueItemDeleteEnabled())
|
foreach (INotificationProvider provider in _notificationFactory.OnQueueItemDeletedEnabled())
|
||||||
{
|
{
|
||||||
try
|
try
|
||||||
{
|
{
|
||||||
await provider.OnQueueItemDelete(notification);
|
await provider.OnQueueItemDeleted(notification);
|
||||||
|
}
|
||||||
|
catch (Exception exception)
|
||||||
|
{
|
||||||
|
_logger.LogWarning(exception, "failed to send notification | provider {provider}", provider.Name);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
public async Task Notify(DownloadCleanedNotification notification)
|
||||||
|
{
|
||||||
|
foreach (INotificationProvider provider in _notificationFactory.OnDownloadCleanedEnabled())
|
||||||
|
{
|
||||||
|
try
|
||||||
|
{
|
||||||
|
await provider.OnDownloadCleaned(notification);
|
||||||
}
|
}
|
||||||
catch (Exception exception)
|
catch (Exception exception)
|
||||||
{
|
{
|
||||||
|
|||||||
@@ -5,6 +5,7 @@ using Domain.Enums;
|
|||||||
using Domain.Models.Arr;
|
using Domain.Models.Arr;
|
||||||
using Domain.Models.Arr.Queue;
|
using Domain.Models.Arr.Queue;
|
||||||
using Infrastructure.Verticals.Arr;
|
using Infrastructure.Verticals.Arr;
|
||||||
|
using Infrastructure.Verticals.Arr.Interfaces;
|
||||||
using Infrastructure.Verticals.Context;
|
using Infrastructure.Verticals.Context;
|
||||||
using Infrastructure.Verticals.DownloadClient;
|
using Infrastructure.Verticals.DownloadClient;
|
||||||
using Infrastructure.Verticals.Jobs;
|
using Infrastructure.Verticals.Jobs;
|
||||||
@@ -48,7 +49,7 @@ public sealed class QueueCleaner : GenericHandler
|
|||||||
using var _ = LogContext.PushProperty("InstanceName", instanceType.ToString());
|
using var _ = LogContext.PushProperty("InstanceName", instanceType.ToString());
|
||||||
|
|
||||||
HashSet<SearchItem> itemsToBeRefreshed = [];
|
HashSet<SearchItem> itemsToBeRefreshed = [];
|
||||||
ArrClient arrClient = GetClient(instanceType);
|
IArrClient arrClient = GetClient(instanceType);
|
||||||
|
|
||||||
// push to context
|
// push to context
|
||||||
ContextProvider.Set(nameof(ArrInstance) + nameof(ArrInstance.Url), instance.Url);
|
ContextProvider.Set(nameof(ArrInstance) + nameof(ArrInstance.Url), instance.Url);
|
||||||
@@ -113,7 +114,7 @@ public sealed class QueueCleaner : GenericHandler
|
|||||||
}
|
}
|
||||||
|
|
||||||
await arrClient.DeleteQueueItemAsync(instance, record, removeFromClient);
|
await arrClient.DeleteQueueItemAsync(instance, record, removeFromClient);
|
||||||
await _notifier.NotifyQueueItemDelete(removeFromClient, deleteReason);
|
await _notifier.NotifyQueueItemDeleted(removeFromClient, deleteReason);
|
||||||
}
|
}
|
||||||
});
|
});
|
||||||
|
|
||||||
|
|||||||
@@ -8,6 +8,8 @@ Project("{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}") = "Infrastructure", "Infrastru
|
|||||||
EndProject
|
EndProject
|
||||||
Project("{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}") = "Domain", "Domain\Domain.csproj", "{8871592A-B260-4B15-8EF8-6AB24480DE5D}"
|
Project("{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}") = "Domain", "Domain\Domain.csproj", "{8871592A-B260-4B15-8EF8-6AB24480DE5D}"
|
||||||
EndProject
|
EndProject
|
||||||
|
Project("{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}") = "Infrastructure.Tests", "Infrastructure.Tests\Infrastructure.Tests.csproj", "{F2DBB5FD-8D93-45D0-B211-5E9A8C14B684}"
|
||||||
|
EndProject
|
||||||
Global
|
Global
|
||||||
GlobalSection(SolutionConfigurationPlatforms) = preSolution
|
GlobalSection(SolutionConfigurationPlatforms) = preSolution
|
||||||
Debug|Any CPU = Debug|Any CPU
|
Debug|Any CPU = Debug|Any CPU
|
||||||
@@ -30,5 +32,9 @@ Global
|
|||||||
{8871592A-B260-4B15-8EF8-6AB24480DE5D}.Debug|Any CPU.Build.0 = Debug|Any CPU
|
{8871592A-B260-4B15-8EF8-6AB24480DE5D}.Debug|Any CPU.Build.0 = Debug|Any CPU
|
||||||
{8871592A-B260-4B15-8EF8-6AB24480DE5D}.Release|Any CPU.ActiveCfg = Release|Any CPU
|
{8871592A-B260-4B15-8EF8-6AB24480DE5D}.Release|Any CPU.ActiveCfg = Release|Any CPU
|
||||||
{8871592A-B260-4B15-8EF8-6AB24480DE5D}.Release|Any CPU.Build.0 = Release|Any CPU
|
{8871592A-B260-4B15-8EF8-6AB24480DE5D}.Release|Any CPU.Build.0 = Release|Any CPU
|
||||||
|
{F2DBB5FD-8D93-45D0-B211-5E9A8C14B684}.Debug|Any CPU.ActiveCfg = Debug|Any CPU
|
||||||
|
{F2DBB5FD-8D93-45D0-B211-5E9A8C14B684}.Debug|Any CPU.Build.0 = Debug|Any CPU
|
||||||
|
{F2DBB5FD-8D93-45D0-B211-5E9A8C14B684}.Release|Any CPU.ActiveCfg = Release|Any CPU
|
||||||
|
{F2DBB5FD-8D93-45D0-B211-5E9A8C14B684}.Release|Any CPU.Build.0 = Release|Any CPU
|
||||||
EndGlobalSection
|
EndGlobalSection
|
||||||
EndGlobal
|
EndGlobal
|
||||||
|
|||||||
@@ -1,2 +0,0 @@
|
|||||||
{
|
|
||||||
}
|
|
||||||
@@ -148,6 +148,8 @@ services:
|
|||||||
- ./data/lidarr/config:/config
|
- ./data/lidarr/config:/config
|
||||||
- ./data/lidarr/music:/music
|
- ./data/lidarr/music:/music
|
||||||
- ./data/qbittorrent/downloads:/downloads
|
- ./data/qbittorrent/downloads:/downloads
|
||||||
|
# - ./data/deluge/downloads:/downloads
|
||||||
|
# - ./data/transmission/downloads:/downloads
|
||||||
ports:
|
ports:
|
||||||
- 8686:8686
|
- 8686:8686
|
||||||
restart: unless-stopped
|
restart: unless-stopped
|
||||||
@@ -163,6 +165,8 @@ services:
|
|||||||
- ./data/readarr/config:/config
|
- ./data/readarr/config:/config
|
||||||
- ./data/readarr/books:/books
|
- ./data/readarr/books:/books
|
||||||
- ./data/qbittorrent/downloads:/downloads
|
- ./data/qbittorrent/downloads:/downloads
|
||||||
|
# - ./data/deluge/downloads:/downloads
|
||||||
|
# - ./data/transmission/downloads:/downloads
|
||||||
ports:
|
ports:
|
||||||
- 8787:8787
|
- 8787:8787
|
||||||
restart: unless-stopped
|
restart: unless-stopped
|
||||||
@@ -171,6 +175,8 @@ services:
|
|||||||
image: ghcr.io/flmorg/cleanuperr:latest
|
image: ghcr.io/flmorg/cleanuperr:latest
|
||||||
container_name: cleanuperr
|
container_name: cleanuperr
|
||||||
environment:
|
environment:
|
||||||
|
- DRY_RUN=false
|
||||||
|
|
||||||
- LOGGING__LOGLEVEL=Debug
|
- LOGGING__LOGLEVEL=Debug
|
||||||
- LOGGING__FILE__ENABLED=true
|
- LOGGING__FILE__ENABLED=true
|
||||||
- LOGGING__FILE__PATH=/var/logs
|
- LOGGING__FILE__PATH=/var/logs
|
||||||
@@ -181,6 +187,7 @@ services:
|
|||||||
|
|
||||||
- TRIGGERS__QUEUECLEANER=0/30 * * * * ?
|
- TRIGGERS__QUEUECLEANER=0/30 * * * * ?
|
||||||
- TRIGGERS__CONTENTBLOCKER=0/30 * * * * ?
|
- TRIGGERS__CONTENTBLOCKER=0/30 * * * * ?
|
||||||
|
- TRIGGERS__DOWNLOADCLEANER=0/30 * * * * ?
|
||||||
|
|
||||||
- QUEUECLEANER__ENABLED=true
|
- QUEUECLEANER__ENABLED=true
|
||||||
- QUEUECLEANER__RUNSEQUENTIALLY=true
|
- QUEUECLEANER__RUNSEQUENTIALLY=true
|
||||||
@@ -196,17 +203,28 @@ services:
|
|||||||
- CONTENTBLOCKER__IGNORE_PRIVATE=true
|
- CONTENTBLOCKER__IGNORE_PRIVATE=true
|
||||||
- CONTENTBLOCKER__DELETE_PRIVATE=false
|
- CONTENTBLOCKER__DELETE_PRIVATE=false
|
||||||
|
|
||||||
|
- DOWNLOADCLEANER__ENABLED=true
|
||||||
|
- DOWNLOADCLEANER__DELETE_PRIVATE=false
|
||||||
|
- DOWNLOADCLEANER__CATEGORIES__0__NAME=tv-sonarr
|
||||||
|
- DOWNLOADCLEANER__CATEGORIES__0__MAX_RATIO=-1
|
||||||
|
- DOWNLOADCLEANER__CATEGORIES__0__MIN_SEED_TIME=0
|
||||||
|
- DOWNLOADCLEANER__CATEGORIES__0__MAX_SEED_TIME=0.01
|
||||||
|
- DOWNLOADCLEANER__CATEGORIES__1__NAME=radarr
|
||||||
|
- DOWNLOADCLEANER__CATEGORIES__1__MAX_RATIO=-1
|
||||||
|
- DOWNLOADCLEANER__CATEGORIES__1__MIN_SEED_TIME=0
|
||||||
|
- DOWNLOADCLEANER__CATEGORIES__1__MAX_SEED_TIME=0.01
|
||||||
|
|
||||||
- DOWNLOAD_CLIENT=qbittorrent
|
- DOWNLOAD_CLIENT=qbittorrent
|
||||||
- QBITTORRENT__URL=http://qbittorrent:8080
|
- QBITTORRENT__URL=http://qbittorrent:8080
|
||||||
- QBITTORRENT__USERNAME=test
|
- QBITTORRENT__USERNAME=test
|
||||||
- QBITTORRENT__PASSWORD=testing
|
- QBITTORRENT__PASSWORD=testing
|
||||||
# OR
|
# OR
|
||||||
# - DOWNLOAD_CLIENT=deluge
|
# - DOWNLOAD_CLIENT=deluge
|
||||||
# - DELUGE__URL=http://localhost:8112
|
# - DELUGE__URL=http://deluge:8112
|
||||||
# - DELUGE__PASSWORD=testing
|
# - DELUGE__PASSWORD=testing
|
||||||
# OR
|
# OR
|
||||||
# - DOWNLOAD_CLIENT=transmission
|
# - DOWNLOAD_CLIENT=transmission
|
||||||
# - TRANSMISSION__URL=http://localhost:9091
|
# - TRANSMISSION__URL=http://transmission:9091
|
||||||
# - TRANSMISSION__USERNAME=test
|
# - TRANSMISSION__USERNAME=test
|
||||||
# - TRANSMISSION__PASSWORD=testing
|
# - TRANSMISSION__PASSWORD=testing
|
||||||
|
|
||||||
@@ -231,7 +249,7 @@ services:
|
|||||||
|
|
||||||
# - NOTIFIARR__ON_IMPORT_FAILED_STRIKE=true
|
# - NOTIFIARR__ON_IMPORT_FAILED_STRIKE=true
|
||||||
# - NOTIFIARR__ON_STALLED_STRIKE=true
|
# - NOTIFIARR__ON_STALLED_STRIKE=true
|
||||||
# - NOTIFIARR__ON_QUEUE_ITEM_DELETE=true
|
# - NOTIFIARR__ON_QUEUE_ITEM_DELETED=true
|
||||||
# - NOTIFIARR__API_KEY=notifiarr_secret
|
# - NOTIFIARR__API_KEY=notifiarr_secret
|
||||||
# - NOTIFIARR__CHANNEL_ID=discord_channel_id
|
# - NOTIFIARR__CHANNEL_ID=discord_channel_id
|
||||||
volumes:
|
volumes:
|
||||||
|
|||||||
+519
-6
@@ -1,11 +1,524 @@
|
|||||||
## LOGGING__ENHANCED
|
## Table of contents
|
||||||
|
- [General settings](variables.md#general-settings)
|
||||||
|
- [Queue Cleaner settings](variables.md#queue-cleaner-settings)
|
||||||
|
- [Content Blocker settings](variables.md#content-blocker-settings)
|
||||||
|
- [Download Cleaner settings](variables.md#download-cleaner-settings)
|
||||||
|
- [Download Client settings](variables.md#download-client-settings)
|
||||||
|
- [Arr settings](variables.md#arr-settings)
|
||||||
|
- [Notification settings](variables.md#notification-settings)
|
||||||
|
- [Advanced settings](variables.md#advanced-settings)
|
||||||
|
|
||||||
Some logs may contain information that is hard to read. Enhancing these logs usually comes with the cost of additional calls to the APIs.
|
#
|
||||||
|
|
||||||
If enabled, logs like this
|
### General settings
|
||||||
|
|
||||||
```movie search triggered | http://localhost:7878/ | movie ids: 1, 2```
|
**`DRY_RUN`**
|
||||||
|
- When enabled, simulates irreversible operations (like deletions and notifications) without making actual changes.
|
||||||
|
- Type: Boolean.
|
||||||
|
- Possible values: `true`, `false`.
|
||||||
|
- Default: `false`.
|
||||||
|
- Required: No.
|
||||||
|
|
||||||
will transform into
|
**`LOGGING__LOGLEVEL`**
|
||||||
|
- Controls the detail level of application logs.
|
||||||
|
- Type: String.
|
||||||
|
- Possible values: `Verbose`, `Debug`, `Information`, `Warning`, `Error`, `Fatal`.
|
||||||
|
- Default: `Information`.
|
||||||
|
- Required: No.
|
||||||
|
|
||||||
```movie search triggered | http://localhost:7878/ | [Speak No Evil][The Wild Robot]```
|
**`LOGGING__FILE__ENABLED`**
|
||||||
|
- Enables logging to a file.
|
||||||
|
- Type: Boolean.
|
||||||
|
- Possible values: `true`, `false`.
|
||||||
|
- Default: `false`.
|
||||||
|
- Required: No.
|
||||||
|
|
||||||
|
**`LOGGING__FILE__PATH`**
|
||||||
|
- Directory where log files will be saved.
|
||||||
|
- Type: String.
|
||||||
|
- Default: Empty.
|
||||||
|
- Required: No.
|
||||||
|
|
||||||
|
**`LOGGING__ENHANCED`**
|
||||||
|
- Provides more detailed descriptions in logs whenever possible.
|
||||||
|
- Type: Boolean.
|
||||||
|
- Possible values: `true`, `false`.
|
||||||
|
- Default: `true`.
|
||||||
|
- Required: No.
|
||||||
|
</details>
|
||||||
|
|
||||||
|
#
|
||||||
|
|
||||||
|
### Queue Cleaner settings
|
||||||
|
|
||||||
|
**`TRIGGERS__QUEUECLEANER`**
|
||||||
|
- Cron schedule for the queue cleaner job.
|
||||||
|
- Type: String - [Quartz cron format](https://www.quartz-scheduler.org/documentation/quartz-2.3.0/tutorials/crontrigger.html).
|
||||||
|
- Default: `0 0/5 * * * ?` (every 5 minutes).
|
||||||
|
- Required: Yes if queue cleaner is enabled.
|
||||||
|
|
||||||
|
> [!NOTE]
|
||||||
|
> - Maximum interval is 6 hours.
|
||||||
|
> - Ignored if `QUEUECLEANER__RUNSEQUENTIALLY=true` and `CONTENTBLOCKER__ENABLED=true`.
|
||||||
|
|
||||||
|
**`QUEUECLEANER__ENABLED`**
|
||||||
|
- Enables or disables the queue cleaning functionality.
|
||||||
|
- When enabled, processes all items in the *arr queue.
|
||||||
|
- Type: Boolean
|
||||||
|
- Possible values: `true`, `false`
|
||||||
|
- Default: `true`
|
||||||
|
- Required: No.
|
||||||
|
|
||||||
|
**`QUEUECLEANER__RUNSEQUENTIALLY`**
|
||||||
|
- Controls whether queue cleaner runs after content blocker instead of in parallel.
|
||||||
|
- When `true`, streamlines the cleaning process by running immediately after content blocker.
|
||||||
|
- Type: Boolean
|
||||||
|
- Possible values: `true`, `false`
|
||||||
|
- Default: `true`
|
||||||
|
- Required: No.
|
||||||
|
|
||||||
|
**`QUEUECLEANER__IMPORT_FAILED_MAX_STRIKES`**
|
||||||
|
- Number of strikes before removing a failed import.
|
||||||
|
- Set to `0` to never remove failed imports.
|
||||||
|
- A strike is given when an item is stalled, stuck in metadata downloading, or failed to be imported.
|
||||||
|
- Type: Integer
|
||||||
|
- Possible values: `0` or greater
|
||||||
|
- Default: `0`
|
||||||
|
- Required: No.
|
||||||
|
|
||||||
|
**`QUEUECLEANER__IMPORT_FAILED_IGNORE_PRIVATE`**
|
||||||
|
- Controls whether to ignore failed imports from private trackers.
|
||||||
|
- Type: Boolean
|
||||||
|
- Possible values: `true`, `false`
|
||||||
|
- Default: `false`
|
||||||
|
- Required: No.
|
||||||
|
|
||||||
|
**`QUEUECLEANER__IMPORT_FAILED_DELETE_PRIVATE`**
|
||||||
|
- Controls whether to delete failed imports from private trackers from the download client.
|
||||||
|
- Has no effect if `QUEUECLEANER__IMPORT_FAILED_IGNORE_PRIVATE` is `true`.
|
||||||
|
- Type: Boolean
|
||||||
|
- Possible values: `true`, `false`
|
||||||
|
- Default: `false`
|
||||||
|
- Required: No.
|
||||||
|
|
||||||
|
> [!WARNING]
|
||||||
|
> Setting `QUEUECLEANER__IMPORT_FAILED_DELETE_PRIVATE=true` means you don't care about seeding, ratio, H&R and potentially losing your tracker account.
|
||||||
|
|
||||||
|
**`QUEUECLEANER__IMPORT_FAILED_IGNORE_PATTERNS`**
|
||||||
|
- Patterns to look for in failed import messages that should be ignored.
|
||||||
|
- Multiple patterns can be specified using incrementing numbers starting from 0.
|
||||||
|
- Type: String array
|
||||||
|
- Default: Empty.
|
||||||
|
- Required: No.
|
||||||
|
- Example:
|
||||||
|
```yaml
|
||||||
|
QUEUECLEANER__IMPORT_FAILED_IGNORE_PATTERNS__0: "title mismatch"
|
||||||
|
QUEUECLEANER__IMPORT_FAILED_IGNORE_PATTERNS__1: "manual import required"
|
||||||
|
```
|
||||||
|
|
||||||
|
**`QUEUECLEANER__STALLED_MAX_STRIKES`**
|
||||||
|
- Number of strikes before removing a stalled download.
|
||||||
|
- Set to `0` to never remove stalled downloads.
|
||||||
|
- A strike is given when download speed is 0.
|
||||||
|
- Type: Integer
|
||||||
|
- Possible values: `0` or greater
|
||||||
|
- Default: `0`
|
||||||
|
- Required: No.
|
||||||
|
|
||||||
|
**`QUEUECLEANER__STALLED_RESET_STRIKES_ON_PROGRESS`**
|
||||||
|
- Controls whether to remove strikes if any download progress was made since last checked.
|
||||||
|
- Type: Boolean
|
||||||
|
- Possible values: `true`, `false`
|
||||||
|
- Default: `false`
|
||||||
|
- Required: No.
|
||||||
|
|
||||||
|
**`QUEUECLEANER__STALLED_IGNORE_PRIVATE`**
|
||||||
|
- Controls whether to ignore stalled downloads from private trackers.
|
||||||
|
- Type: Boolean
|
||||||
|
- Possible values: `true`, `false`
|
||||||
|
- Default: `false`
|
||||||
|
- Required: No.
|
||||||
|
|
||||||
|
**`QUEUECLEANER__STALLED_DELETE_PRIVATE`**
|
||||||
|
- Controls whether to delete stalled private downloads from the download client.
|
||||||
|
- Has no effect if `QUEUECLEANER__STALLED_IGNORE_PRIVATE` is `true`.
|
||||||
|
- Type: Boolean
|
||||||
|
- Possible values: `true`, `false`
|
||||||
|
- Default: `false`
|
||||||
|
- Required: No.
|
||||||
|
|
||||||
|
> [!WARNING]
|
||||||
|
> Setting `QUEUECLEANER__STALLED_DELETE_PRIVATE=true` means you don't care about seeding, ratio, H&R and potentially losing your tracker account.
|
||||||
|
|
||||||
|
#
|
||||||
|
|
||||||
|
### Content Blocker settings
|
||||||
|
|
||||||
|
**`TRIGGERS__CONTENTBLOCKER`**
|
||||||
|
- Cron schedule for the content blocker job.
|
||||||
|
- Type: String - [Quartz cron format](https://www.quartz-scheduler.org/documentation/quartz-2.3.0/tutorials/crontrigger.html).
|
||||||
|
- Default: `0 0/5 * * * ?` (every 5 minutes).
|
||||||
|
- Required: No.
|
||||||
|
|
||||||
|
> [!NOTE]
|
||||||
|
> - Maximum interval is 6 hours.
|
||||||
|
|
||||||
|
**`CONTENTBLOCKER__ENABLED`**
|
||||||
|
- Enables or disables the content blocker functionality.
|
||||||
|
- When enabled, processes all items in the *arr queue and marks unwanted files.
|
||||||
|
- Type: Boolean
|
||||||
|
- Possible values: `true`, `false`
|
||||||
|
- Default: `false`
|
||||||
|
- Required: No.
|
||||||
|
|
||||||
|
**`CONTENTBLOCKER__IGNORE_PRIVATE`**
|
||||||
|
- Controls whether to ignore downloads from private trackers.
|
||||||
|
- Type: Boolean
|
||||||
|
- Possible values: `true`, `false`
|
||||||
|
- Default: `false`
|
||||||
|
- Required: No.
|
||||||
|
|
||||||
|
**`CONTENTBLOCKER__DELETE_PRIVATE`**
|
||||||
|
- Controls whether to delete private downloads that have all files blocked from the download client.
|
||||||
|
- Has no effect if `CONTENTBLOCKER__IGNORE_PRIVATE` is `true`.
|
||||||
|
- Type: Boolean
|
||||||
|
- Possible values: `true`, `false`
|
||||||
|
- Default: `false`
|
||||||
|
- Required: No.
|
||||||
|
|
||||||
|
> [!WARNING]
|
||||||
|
> Setting `CONTENTBLOCKER__DELETE_PRIVATE=true` means you don't care about seeding, ratio, H&R and potentially losing your tracker account.
|
||||||
|
|
||||||
|
#
|
||||||
|
|
||||||
|
### Download Cleaner settings
|
||||||
|
|
||||||
|
**`TRIGGERS__DOWNLOADCLEANER`**
|
||||||
|
- Cron schedule for the download cleaner job.
|
||||||
|
- Type: String - [Quartz cron format](https://www.quartz-scheduler.org/documentation/quartz-2.3.0/tutorials/crontrigger.html).
|
||||||
|
- Default: `0 0 * * * ?` (every hour).
|
||||||
|
- Required: No.
|
||||||
|
|
||||||
|
> [!NOTE]
|
||||||
|
> - Maximum interval is 6 hours.
|
||||||
|
|
||||||
|
**`DOWNLOADCLEANER__ENABLED`**
|
||||||
|
- Enables or disables the download cleaner functionality.
|
||||||
|
- When enabled, automatically cleans up downloads that have been seeding for a certain amount of time.
|
||||||
|
- Type: Boolean.
|
||||||
|
- Possible values: `true`, `false`
|
||||||
|
- Default: `false`
|
||||||
|
- Required: No.
|
||||||
|
|
||||||
|
**`DOWNLOADCLEANER__DELETE_PRIVATE`**
|
||||||
|
- Controls whether to delete private downloads.
|
||||||
|
- Type: Boolean.
|
||||||
|
- Possible values: `true`, `false`
|
||||||
|
- Default: `false`
|
||||||
|
- Required: No.
|
||||||
|
|
||||||
|
> [!WARNING]
|
||||||
|
> Setting `DOWNLOADCLEANER__DELETE_PRIVATE=true` means you don't care about seeding, ratio, H&R and potentially losing your tracker account.
|
||||||
|
|
||||||
|
**`DOWNLOADCLEANER__CATEGORIES__0__NAME`**
|
||||||
|
- Name of the category to clean.
|
||||||
|
- Type: String.
|
||||||
|
- Default: Empty.
|
||||||
|
- Required: No.
|
||||||
|
|
||||||
|
> [!NOTE]
|
||||||
|
> The category name must match the category that was set in the *arr.
|
||||||
|
> For qBittorrent, the category name is the name of the download category.
|
||||||
|
> For Deluge, the category name is the name of the label.
|
||||||
|
> For Transmission, the category name is the name of the download location.
|
||||||
|
|
||||||
|
**`DOWNLOADCLEANER__CATEGORIES__0__MAX_RATIO`**
|
||||||
|
- Maximum ratio to reach before removing a download.
|
||||||
|
- Type: Decimal.
|
||||||
|
- Possible values: `-1` or greater (`-1` means no limit or disabled).
|
||||||
|
- Default: `-1`
|
||||||
|
- Required: No.
|
||||||
|
|
||||||
|
**`DOWNLOADCLEANER__CATEGORIES__0__MIN_SEED_TIME`**
|
||||||
|
- Minimum number of hours to seed before removing a download, if the ratio has been met.
|
||||||
|
- Used with `MAX_RATIO` to ensure a minimum seed time.
|
||||||
|
- Type: Decimal.
|
||||||
|
- Possible values: `0` or greater.
|
||||||
|
- Default: `0`
|
||||||
|
- Required: No.
|
||||||
|
|
||||||
|
**`DOWNLOADCLEANER__CATEGORIES__0__MAX_SEED_TIME`**
|
||||||
|
- Maximum number of hours to seed before removing a download.
|
||||||
|
- Type: Decimal.
|
||||||
|
- Possible values: `-1` or greater (`-1` means no limit or disabled).
|
||||||
|
- Default: `-1`
|
||||||
|
- Required: No.
|
||||||
|
|
||||||
|
> [!NOTE]
|
||||||
|
> A download is cleaned when any of (`MAX_RATIO` & `MIN_SEED_TIME`) or `MAX_SEED_TIME` is reached.
|
||||||
|
|
||||||
|
> [!NOTE]
|
||||||
|
> Multiple categories can be specified using this format, where `<NUMBER>` starts from 0:
|
||||||
|
> ```yaml
|
||||||
|
> DOWNLOADCLEANER__CATEGORIES__<NUMBER>__NAME
|
||||||
|
> DOWNLOADCLEANER__CATEGORIES__<NUMBER>__MAX_RATIO
|
||||||
|
> DOWNLOADCLEANER__CATEGORIES__<NUMBER>__MIN_SEED_TIME
|
||||||
|
> DOWNLOADCLEANER__CATEGORIES__<NUMBER>__MAX_SEED_TIME
|
||||||
|
> ```
|
||||||
|
|
||||||
|
#
|
||||||
|
|
||||||
|
### Download Client settings
|
||||||
|
|
||||||
|
**`DOWNLOAD_CLIENT`**
|
||||||
|
- Specifies which download client is used by *arrs.
|
||||||
|
- Type: String.
|
||||||
|
- Possible values: `none`, `qbittorrent`, `deluge`, `transmission`.
|
||||||
|
- Default: `none`
|
||||||
|
- Required: No.
|
||||||
|
|
||||||
|
> [!NOTE]
|
||||||
|
> Only one download client can be enabled at a time. If you have more than one download client, you should deploy multiple instances of cleanuperr.
|
||||||
|
|
||||||
|
**`QBITTORRENT__URL`**
|
||||||
|
- URL of the qBittorrent instance.
|
||||||
|
- Type: String.
|
||||||
|
- Default: `http://localhost:8080`.
|
||||||
|
- Required: No.
|
||||||
|
|
||||||
|
**`QBITTORRENT__USERNAME`**
|
||||||
|
- Username for qBittorrent authentication.
|
||||||
|
- Type: String.
|
||||||
|
- Default: Empty.
|
||||||
|
- Required: No.
|
||||||
|
|
||||||
|
**`QBITTORRENT__PASSWORD`**
|
||||||
|
- Password for qBittorrent authentication.
|
||||||
|
- Type: String.
|
||||||
|
- Default: Empty.
|
||||||
|
- Required: No.
|
||||||
|
|
||||||
|
**`DELUGE__URL`**
|
||||||
|
- URL of the Deluge instance.
|
||||||
|
- Type: String.
|
||||||
|
- Default: `http://localhost:8112`.
|
||||||
|
- Required: No.
|
||||||
|
|
||||||
|
**`DELUGE__PASSWORD`**
|
||||||
|
- Password for Deluge authentication.
|
||||||
|
- Type: String.
|
||||||
|
- Default: Empty.
|
||||||
|
- Required: No.
|
||||||
|
|
||||||
|
**`TRANSMISSION__URL`**
|
||||||
|
- URL of the Transmission instance.
|
||||||
|
- Type: String.
|
||||||
|
- Default: `http://localhost:9091`.
|
||||||
|
- Required: No.
|
||||||
|
|
||||||
|
**`TRANSMISSION__USERNAME`**
|
||||||
|
- Username for Transmission authentication.
|
||||||
|
- Type: String.
|
||||||
|
- Default: Empty.
|
||||||
|
- Required: No.
|
||||||
|
|
||||||
|
**`TRANSMISSION__PASSWORD`**
|
||||||
|
- Password for Transmission authentication.
|
||||||
|
- Type: String.
|
||||||
|
- Default: Empty.
|
||||||
|
- Required: No.
|
||||||
|
|
||||||
|
#
|
||||||
|
|
||||||
|
### Arr settings
|
||||||
|
|
||||||
|
**`SONARR__ENABLED`**
|
||||||
|
- Enables or disables Sonarr cleanup.
|
||||||
|
- Type: Boolean
|
||||||
|
- Possible values: `true`, `false`
|
||||||
|
- Default: `false`
|
||||||
|
- Required: No.
|
||||||
|
|
||||||
|
**`SONARR__BLOCK__TYPE`**
|
||||||
|
- Determines how file blocking works for Sonarr.
|
||||||
|
- Type: String
|
||||||
|
- Possible values: `blacklist`, `whitelist`
|
||||||
|
- Default: `blacklist`
|
||||||
|
- Required: No.
|
||||||
|
|
||||||
|
**`SONARR__BLOCK__PATH`**
|
||||||
|
- Path to the blocklist file (local file or URL).
|
||||||
|
- Must be JSON compatible.
|
||||||
|
- Type: String
|
||||||
|
- Default: Empty.
|
||||||
|
- Required: No.
|
||||||
|
|
||||||
|
**`SONARR__SEARCHTYPE`**
|
||||||
|
- Determines what to search for after removing a queue item.
|
||||||
|
- Type: String
|
||||||
|
- Possible values: `Episode`, `Season`, `Series`
|
||||||
|
- Default: `Episode`
|
||||||
|
- Required: No.
|
||||||
|
|
||||||
|
**`SONARR__INSTANCES__0__URL`**
|
||||||
|
- URL of the Sonarr instance.
|
||||||
|
- Type: String
|
||||||
|
- Default: `http://localhost:8989`
|
||||||
|
- Required: No.
|
||||||
|
|
||||||
|
**`SONARR__INSTANCES__0__APIKEY`**
|
||||||
|
- API key for the Sonarr instance.
|
||||||
|
- Type: String
|
||||||
|
- Default: Empty.
|
||||||
|
- Required: No.
|
||||||
|
|
||||||
|
**`RADARR__ENABLED`**
|
||||||
|
- Enables or disables Radarr cleanup.
|
||||||
|
- Type: Boolean
|
||||||
|
- Possible values: `true`, `false`
|
||||||
|
- Default: `false`
|
||||||
|
- Required: No.
|
||||||
|
|
||||||
|
**`RADARR__BLOCK__TYPE`**
|
||||||
|
- Determines how file blocking works for Radarr.
|
||||||
|
- Type: String
|
||||||
|
- Possible values: `blacklist`, `whitelist`
|
||||||
|
- Default: `blacklist`
|
||||||
|
- Required: No.
|
||||||
|
|
||||||
|
**`RADARR__BLOCK__PATH`**
|
||||||
|
- Path to the blocklist file (local file or URL).
|
||||||
|
- Must be JSON compatible.
|
||||||
|
- Type: String
|
||||||
|
- Default: Empty.
|
||||||
|
- Required: No.
|
||||||
|
|
||||||
|
**`RADARR__INSTANCES__0__URL`**
|
||||||
|
- URL of the Radarr instance.
|
||||||
|
- Type: String
|
||||||
|
- Default: `http://localhost:7878`
|
||||||
|
- Required: No.
|
||||||
|
|
||||||
|
**`RADARR__INSTANCES__0__APIKEY`**
|
||||||
|
- API key for the Radarr instance.
|
||||||
|
- Type: String
|
||||||
|
- Default: Empty.
|
||||||
|
- Required: No.
|
||||||
|
|
||||||
|
**`LIDARR__ENABLED`**
|
||||||
|
- Enables or disables Lidarr cleanup.
|
||||||
|
- Type: Boolean
|
||||||
|
- Possible values: `true`, `false`
|
||||||
|
- Default: `false`
|
||||||
|
- Required: No.
|
||||||
|
|
||||||
|
**`LIDARR__BLOCK__TYPE`**
|
||||||
|
- Determines how file blocking works for Lidarr.
|
||||||
|
- Type: String
|
||||||
|
- Possible values: `blacklist`, `whitelist`
|
||||||
|
- Default: `blacklist`
|
||||||
|
- Required: No.
|
||||||
|
|
||||||
|
**`LIDARR__BLOCK__PATH`**
|
||||||
|
- Path to the blocklist file (local file or URL).
|
||||||
|
- Must be JSON compatible.
|
||||||
|
- Type: String
|
||||||
|
- Default: Empty.
|
||||||
|
- Required: No.
|
||||||
|
|
||||||
|
**`LIDARR__INSTANCES__0__URL`**
|
||||||
|
- URL of the Lidarr instance.
|
||||||
|
- Type: String
|
||||||
|
- Default: `http://localhost:8686`
|
||||||
|
- Required: No.
|
||||||
|
|
||||||
|
**`LIDARR__INSTANCES__0__APIKEY`**
|
||||||
|
- API key for the Lidarr instance.
|
||||||
|
- Type: String
|
||||||
|
- Default: Empty.
|
||||||
|
- Required: No.
|
||||||
|
|
||||||
|
> [!NOTE]
|
||||||
|
> Multiple instances can be specified for each *arr using this format, where `<NUMBER>` starts from 0:
|
||||||
|
> ```yaml
|
||||||
|
> <ARR>__INSTANCES__<NUMBER>__URL
|
||||||
|
> <ARR>__INSTANCES__<NUMBER>__APIKEY
|
||||||
|
> ```
|
||||||
|
|
||||||
|
> [!NOTE]
|
||||||
|
> The blocklists (blacklist/whitelist) support the following patterns:
|
||||||
|
> ```
|
||||||
|
> *example // file name ends with "example"
|
||||||
|
> example* // file name starts with "example"
|
||||||
|
> *example* // file name has "example" in the name
|
||||||
|
> example // file name is exactly the word "example"
|
||||||
|
> regex:<ANY_REGEX> // regex that needs to be marked at the start of the line with "regex:"
|
||||||
|
> ```
|
||||||
|
|
||||||
|
> [!NOTE]
|
||||||
|
> [This blacklist](https://raw.githubusercontent.com/flmorg/cleanuperr/refs/heads/main/blacklist) and [this whitelist](https://raw.githubusercontent.com/flmorg/cleanuperr/refs/heads/main/whitelist) can be used for Sonarr and Radarr, but they are not suitable for other *arrs.
|
||||||
|
|
||||||
|
#
|
||||||
|
|
||||||
|
### Notification settings
|
||||||
|
|
||||||
|
**`NOTIFIARR__API_KEY`**
|
||||||
|
- Notifiarr API key for sending notifications.
|
||||||
|
- Requires Notifiarr's [`Passthrough`](https://notifiarr.wiki/en/Website/Integrations/Passthrough) integration to work.
|
||||||
|
- Type: String
|
||||||
|
- Default: Empty.
|
||||||
|
- Required: No.
|
||||||
|
|
||||||
|
**`NOTIFIARR__CHANNEL_ID`**
|
||||||
|
- Discord channel ID where notifications will be sent.
|
||||||
|
- Type: String
|
||||||
|
- Default: Empty.
|
||||||
|
- Required: No.
|
||||||
|
|
||||||
|
**`NOTIFIARR__ON_IMPORT_FAILED_STRIKE`**
|
||||||
|
- Controls whether to notify when an item receives a failed import strike.
|
||||||
|
- Type: Boolean
|
||||||
|
- Possible values: `true`, `false`
|
||||||
|
- Default: `false`
|
||||||
|
- Required: No.
|
||||||
|
|
||||||
|
**`NOTIFIARR__ON_STALLED_STRIKE`**
|
||||||
|
- Controls whether to notify when an item receives a stalled download strike.
|
||||||
|
- Type: Boolean
|
||||||
|
- Possible values: `true`, `false`
|
||||||
|
- Default: `false`
|
||||||
|
- Required: No.
|
||||||
|
|
||||||
|
**`NOTIFIARR__ON_QUEUE_ITEM_DELETED`**
|
||||||
|
- Controls whether to notify when a queue item is deleted.
|
||||||
|
- Type: Boolean
|
||||||
|
- Possible values: `true`, `false`
|
||||||
|
- Default: `false`
|
||||||
|
- Required: No.
|
||||||
|
|
||||||
|
**`NOTIFIARR__ON_DOWNLOAD_CLEANED`**
|
||||||
|
- Controls whether to notify when a download is cleaned.
|
||||||
|
- Type: Boolean
|
||||||
|
- Possible values: `true`, `false`
|
||||||
|
- Default: `false`
|
||||||
|
- Required: No.
|
||||||
|
|
||||||
|
#
|
||||||
|
|
||||||
|
### Advanced settings
|
||||||
|
|
||||||
|
**`HTTP_MAX_RETRIES`**
|
||||||
|
- The number of times to retry a failed HTTP call.
|
||||||
|
- Applies to calls to *arrs, download clients, and other services.
|
||||||
|
- Type: Integer
|
||||||
|
- Possible values: `0` or greater
|
||||||
|
- Default: `0`
|
||||||
|
- Required: No.
|
||||||
|
|
||||||
|
**`HTTP_TIMEOUT`**
|
||||||
|
- The number of seconds to wait before failing an HTTP call.
|
||||||
|
- Applies to calls to *arrs, download clients, and other services.
|
||||||
|
- Type: Integer
|
||||||
|
- Possible values: Greater than `0`
|
||||||
|
- Default: `100`
|
||||||
|
- Required: No.
|
||||||
Reference in New Issue
Block a user