• Joined on 2023-07-25
pta created pull request gg/WeatherNews.jl#11 2024-05-09 17:36:49 +00:00
Various fixes and and a new cron job to ensure consistent video data collection.
pta pushed to main at pta/WeatherNews.jl 2024-05-09 16:47:06 +00:00
cb105176b2 Add cron job to make final view count update after exactly 10 days
bca7133795 Ensure every WNL row in schedule table gets assigned its video
Compare 2 commits »
pta pushed to main at pta/WeatherNews.jl 2024-05-09 16:42:09 +00:00
aa30f399d8 Add cron job to make final view count update after exactly 10 days
101d73b00a Ensure every entry in timetable.json gets assigned its video
Compare 2 commits »
pta pushed to main at pta/WeatherNews.jl 2024-05-09 15:56:31 +00:00
bf92b5c1ee Add cron job to make final view count update after exactly 10 days
b6d7d51f45 Ensure every entry in timetable.json gets assigned its video
Compare 2 commits »
pta pushed to main at pta/WeatherNews.jl 2024-05-09 09:27:48 +00:00
9493ca7ae7 Ensure any and every entry in timetable.json gets assigned its video
ebff641a0a Bump youtube streams request up to 30
Compare 2 commits »
pta pushed to main at pta/WeatherNews.jl 2024-05-08 22:29:43 +00:00
e2754fb4c5 Add PlotlyJS as a dependency
41f8cfd0ae Transform jst into ZonedDateTimes
4fb138e52b Add WeatherNews.Helpers
6d03c43365 Rename seg.n AS segment instead of title
2fc0a8abf5 Insert au PAY shows for 2024-05
Compare 7 commits »
pta pushed to main at pta/WeatherNews.jl 2024-04-26 19:11:22 +00:00
6428bae369 Add Plots as a dependency
f3429e0101 Filter out view_count == 0
9815bc259d Add DB.load_schedule_joined(db)
2bf95ca33a Add some queries that explore schedule.view_count
172c25be6c Add DataFramesMeta as a dependency
Compare 8 commits »
pta commented on pull request gg/WeatherNews.jl#10 2024-04-24 02:20:14 +00:00
Mention yt-dlp in README and start recording view counts

I think the simplest thing to do would be to have a cron job generate plot pngs with every database update:

  1. a single plot on the main page with all rows' views.
  2. a plot on the caster page…
pta commented on pull request gg/WeatherNews.jl#10 2024-04-24 01:28:28 +00:00
Mention yt-dlp in README and start recording view counts

*pyplot
I meant gnuplot, which lots of languages have bindings to, I think.

pta commented on pull request gg/WeatherNews.jl#10 2024-04-24 01:26:45 +00:00
Mention yt-dlp in README and start recording view counts

Great. I've been wanting to collect enough data to start doing some fun statistics. What I really want to do is get graphs on the site to make it easy for the guys to see this analysis. I'm not…

pta commented on pull request gg/WeatherNews.jl#10 2024-04-23 22:49:38 +00:00
Mention yt-dlp in README and start recording view counts

I didn't test this new version. For some reason it's not finding the functions in WeatherNews.jl. Even if I move the script to bin/ with the others

pta commented on pull request gg/WeatherNews.jl#10 2024-04-23 22:32:09 +00:00
Mention yt-dlp in README and start recording view counts
using SQLite, JSON3
using WeatherNews

db = SQLite.DB("/tmp/wn.db")
rows = DBInterface.execute(db,
                           """
                           SELECT id, video_id…
pta commented on pull request gg/WeatherNews.jl#10 2024-04-23 18:19:22 +00:00
Mention yt-dlp in README and start recording view counts

I should have cached the yt-dlp json responses for testing my updated script without spamming youtube's servers again. Would run way faster too, of course.

pta commented on pull request gg/WeatherNews.jl#10 2024-04-23 18:07:07 +00:00
Mention yt-dlp in README and start recording view counts

I should make the script more elaborate by making it check that the video title passes my iswnl function and it's date and segment in the title matches what's stored in the database, and just…

pta commented on pull request gg/WeatherNews.jl#10 2024-04-23 17:48:24 +00:00
Mention yt-dlp in README and start recording view counts

https://www.youtube.com/watch?v=SwVz3Oni58M

"live_status": "not_live",

This is the problem. That URL points to the second calendar-making video!, not Yui's Moon that day, so it was…

pta commented on pull request gg/WeatherNews.jl#10 2024-04-23 17:34:45 +00:00
Mention yt-dlp in README and start recording view counts

the @debug under catch didn't output anything, so I guess the UPDATE "succeeded"

pta commented on pull request gg/WeatherNews.jl#10 2024-04-23 17:28:02 +00:00
Mention yt-dlp in README and start recording view counts
1088  11         4           2023-11-26T14:00:00+09:00  iUHVlwj7Ivo  42068
1089  3          5           2023-11-26T17:00:00+09:00  hWp6FPfTT9A  43370
1090  8          6           2023-11-26T…
pta commented on pull request gg/WeatherNews.jl#10 2024-04-23 17:13:35 +00:00
Mention yt-dlp in README and start recording view counts

finally finished

progress: 1980

real    94m36.556s
user    60m30.666s
sys     5m34.231s
pta commented on pull request gg/WeatherNews.jl#10 2024-04-23 16:05:37 +00:00
Mention yt-dlp in README and start recording view counts

so far so good
sqlite3 /tmp/wn.db 'select * from schedule WHERE segment_id != 8 AND video_id IS NOT NULL AND view_count IS NOT NULL;'

pta commented on pull request gg/WeatherNews.jl#10 2024-04-23 15:40:08 +00:00
Mention yt-dlp in README and start recording view counts

it's actually taking max 2 seconds per update so far