Various fixes and and a new cron job to ensure consistent video data collection.

pta commented

2024-05-09 17:36:48 +00:00

Contributor

I reversed the logic of video ID assignment from "match each video to regular scheduled row in schedule table" to "match each row in schedule table to its video". So now all irregular caster schedules will get their corresponding video data, no matter how they split up the schedule. Rows for segments split between two casters will get the same video IDs and view counts. The error in the attached pic needs to get fixed manually, but such errors shouldn't happen again.

I also want to give all videos a long and fixed amount of time to age before making a final view count record. The new cron job will automatically fix Momo's May 3rd low view count error, and the larger yt-dlp json request will make it less likely to happen.

I changed some db update functions to target by rowid rather than jst, so checks against simultaneous AU pay rows are unnecessary, and locating the row by it's ID should be faster than doing so by string comparisons on the jst column.

Added some type annotations to catch errors at compile time and serve as documentation. Also added a constraint against AU pay streams to find_schedule_by_jst

I reversed the logic of video ID assignment from "match each video to regular scheduled row in schedule table" to "match each row in schedule table to its video". So now all irregular caster schedules will get their corresponding video data, no matter how they split up the schedule. Rows for segments split between two casters will get the same video IDs and view counts. The error in the attached pic needs to get fixed manually, but such errors shouldn't happen again. I also want to give all videos a long and fixed amount of time to age before making a final view count record. The new cron job will automatically fix Momo's May 3rd low view count error, and the larger yt-dlp json request will make it less likely to happen. I changed some db update functions to target by rowid rather than jst, so checks against simultaneous AU pay rows are unnecessary, and locating the row by it's ID should be faster than doing so by string comparisons on the jst column. Added some type annotations to catch errors at compile time and serve as documentation. Also added a constraint against AU pay streams to `find_schedule_by_jst`

240510_02:29:27.png

11 KiB

240510_02:26:24.png

15 KiB

pta added 3 commits 2024-05-09 17:36:49 +00:00

ebff641a0a Bump youtube streams request up to 30

Momo's Sunshine stream on 2024-05-03 got buried under 25 newer streams
just before the cron job could record its view count after it got
unprivatized. A cutoff at 30 should always be ample enough.

bca7133795 Ensure every WNL row in schedule table gets assigned its video

Now every caster schedule entry will get assigned data from its
corresponding video, no matter what unusual scheduled time it has.

Videos are known well advance, so updating every 3 hours, or once per
stream, should be enough to record new streams, even if the youtube json
request fails once or twice.

Upcoming streams are now assigned a null rather than 0 view count so
zeros aren't accidentally included in view count calculations.

Upcoming streams now have their jst derived from their start time rather
than titles, in case staff initially makes a copy-paste error to the
title.

cb105176b2 Add cron job to make final view count update after exactly 10 days

This cron job will run twice per live WNL segment to update the view
count of the stream from exactly 10 days ago. This gives every stream
exactly the same amount of time to mature before we record its view
count for the last time. Ten days gives even notable streams plenty of
time to get most of its views before the rewatch per day rate drops to
nearly nothing.