[ / / / / / / / / / / / / / ] [ dir / 2hu / asmr / choroy / dempart / general / leftyb / mde / vichan ]

/hydrus/ - Hydrus Network

Bug reports, feature requests, and other discussion for the hydrus network.

Catalog   Archive

Winner of the 80rd Attention-Hungry Games
/otter/ - Otter For Your Soul

THE INFINITY CUP IS COMING BACK
May 2019 - 8chan Transparency Report
Name
Email
Subject
Comment *
File
Password (Randomized for file and post deletion; you may also set your own.)
* = required field[▶ Show post options & limits]
Confused? See the FAQ.
Embed
(replaces files and can be used instead)
Options

Allowed file types:jpg, jpeg, gif, png, webm, mp4, swf, pdf
Max filesize is 16 MB.
Max image dimensions are 15000 x 15000.
You may upload 5 per post.


New user? Start here ---> http://hydrusnetwork.github.io/hydrus/

Experienced user with a bit of cash who wants to help out? ---> Patreon

Current to-do list has: 1,744 items

The program is now on Python 3! Check v335 release post if you need to update from before then!

Current big job: Duplicate db overhaul and filter workflow improvements


YouTube embed. Click thumbnail to play.

a413c6  No.12861[Reply]

windows

zip: https://github.com/hydrusnetwork/hydrus/releases/download/v355/Hydrus.Network.355.-.Windows.-.Extract.only.zip

exe: https://github.com/hydrusnetwork/hydrus/releases/download/v355/Hydrus.Network.355.-.Windows.-.Installer.exe

os x

app: https://github.com/hydrusnetwork/hydrus/releases/download/v355/Hydrus.Network.355.-.OS.X.-.App.dmg

linux

tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v355/Hydrus.Network.355.-.Linux.-.Executable.tar.gz

source

tar.gz: https://github.com/hydrusnetwork/hydrus/archive/v355.tar.gz

I had a good week catching up on small jobs.

duplicates

I added a 'trash file' button to the duplicate filter hover window, tucked up in a new row with the cog button so they fit on thinner windows. If both potential files are worth deleting, hitting this is an easy way to dismiss them immediately. I also fixed some over-aggressive potential pairs removal from last week's alternates update. Users with many alternates may have lost some valid intra-alternate-group potential pairs, which I expect to add back in once this whole overhaul is complete.

The duplicate action buttons (and several others in the media viewer) are now aware of thePost too long. Click here to view the full text.

26 posts and 15 image replies omitted. Click reply to view.

919719  No.12945

Ran into a duplicate that I don't want to deal with

long story short, I want to deal with these ones in the threads I got them from through watchers.

Is it possible to have a filter to hold potential dups and not show them again till you flush the filter?




File: b28babaa13b7dbf⋯.png (765.02 KB, 1278x720, 71:40, b28babaa13b7dbfb92d271ee0a….png)

136209  No.12641[Reply]

Here you can ask questions so that the board is not clogged with small threads.

Old thread >>9327

102 posts and 7 image replies omitted. Click reply to view.

f98983  No.12932

>>12930

The pages on pixiv in the browser are fine, they continue as normal. But no matter how many times I refreshed the client, reloaded page 12 in the browser and copied it again, it gave me the previous pages' images over and over, which obviously didn't import because they're already in the db.

Here's the URL I was copying:

https://www.pixiv.net/search.php?word=%E3%81%A9%E3%81%86%E3%81%B6%E3%81%A4%E3%81%AE%E6%A3%AE&s_mode=s_tag_full&order=date&p=12

The previous URLs had exactly the same layout except the number was just lower. 11 is the last one to load properly, after that it just returns page 11 irrespective of the page number.


f3ab0e  No.12933

File: 79a1eb2c07b16ec⋯.jpg (34.62 KB, 832x289, 832:289, 3464634.jpg)

>>12929

Actually forget all that, I'm an idiot. I didn't set the login to shimmie, this probably should be a default setting for rule34hentai.net if it isn't already.


716cb8  No.12934

File: 5d5a0908308f8ca⋯.png (9.62 KB, 367x233, 367:233, Capture.PNG)

Why do I keep getting this?

How do I stop getting this?

I've only got three pages open.


f98983  No.12935

>>12932

I'm dumb and I figured it out. It needed a login to complete more than 11 pages it seems.


9dd526  No.12946

Just started using Hydrus, and I'm trying to get it to autotag an existing collection, but I'm a bit confused.

>Tags are rich, cpu-intensive metadata. My repository has hundreds of millions of mappings, and your client will eventually download and index them all.

Seems way overkill that the user should have to download all that information if they only need tags for the specific files that are part of their own collection. Isn't there a more efficient method?

I'm aware also of the tag archives, but it seems that's also a broad and inefficient approach containing data for the complete file set from a booru/gallery website as well. Is there no other way except to maintain a massive database of all the tag associations of all files ever seen? If downloaders could be used to query downloading tags only for the user's existing files (it may be possible, but I am ignorant), that would be very useful and eliminate the need to waste HDD space/ processing power.




File: 7a56ef4350b0bb9⋯.jpg (64.84 KB, 445x488, 445:488, 7a56ef4350b0bb96c5b6a4eb80….jpg)

ff2c17  No.11542[Reply]

BUGS THREAD

160 posts and 55 image replies omitted. Click reply to view.

495a87  No.12906

File: 832befdba800127⋯.png (4.78 KB, 512x137, 512:137, sankaku.png)

This is less of a bug report, but more of a fix for a longstanding known issue.

So at some point recently Sankakuchan decided to upload every doujin they could get their hands on, which is pretty annoying considering the default hydrus parsers limitations as it effectively buries actual content. Also said doujins are only like 1/3 tagged which is a pain in the ass.

Some light investigation and prototyping and I was able to fix the hydrus parser so it could just keep going forever in theory. I'm honestly surprised this was ever an issue considering how simple the fix was, basically I just made it get the next page correctly.

More specifically an html tag, a div specifically, contained an attribute called 'next-page-url' that contains the url for the next page by id which has no limitation rather than &page which is limited to page 25/50 if authenticated.

Some light testing on something I indexed a while back got through 1390 posts.


4c59b4  No.12910

I've got two images, one of which i've accidentally tagged as a worse quality dupe to the other. Trying to set that image as an alternate image by right clicking on both, Duplicates > Set relationship > Set as Alternates results in both files still being tagged as one worse and one better. Can anyone else confirm?


e12626  No.12915

>>12910

Yeah, although I'm like 95% sure hydrusdev knows, I recall reading something about it like a week ago.

If you're on 355 (possibly 354 too), the duplicate system is in the middle of an overhaul and hydrusdev needs to fix the context menu duplicate commands. I've been having a fun time with cg sets due to this.


e804c6  No.12940

Using v355 on linux. I filter deduplicates bit it doesn't apply anything. After increasing the search distance from 0 to 2 it was still listing all the previous images. I moved to 4 and it still lists the previous images from distances 0 and 2 since the filter didn't apply the commits I guess. It worked previously so v355 broke it.


024cf6  No.12941

>>12940

Same here. I made the mistake of importing multiple CG sets, then scratching my head wondering if I've already sorted their duplicates halfway through.




File: 1426721772716.png (100.78 KB, 1624x1081, 1624:1081, 1327614072601.png)

7f2c0e  No.471[Reply]


Drag and drop windows with tag rules. Show two windows side by side and one window can be programmed with the rule "ADD tag foo" and the other one has the rule "REMOVE tag foo, ADD tag bar" and you can drag and drop files to them.

Deriving tags from regex of other tags/namespace tags. A file has the tag "filename:big_ugly_name" and we could regex that namespace for another tag.

Tag sets with hotkeys: save a set of tags under a hotkey so it's quick to add them to a file while filtering

Opaque window behind tag list in the corner so it doesn't get hidden by picture background

Option to default certain mime types to be excluded from slideshow and only open externally, will help with videos with odd codecs that don't preview in the slideshow correctly

Option to specify hamming distance in "find similar images", you can't change the option once it's in the filter window and you have to enter the hash manually in the "system:similar to" option
693 posts and 199 image replies omitted. Click reply to view.

2353ee  No.12904

In the current system to show tags on top of thumbnails, add the ability to show metadata like filetype, dimensions, file size, duration etc.


b7855f  No.12917

File: e53c958f8799109⋯.png (60.13 KB, 626x1050, 313:525, ClipboardImage.png)

Two suggestions.

First: Optional disjointed duplicated search. It shows you all duplicate relationships that together have all those tags, instead of just a big and on one or both files. Maybe keep it unless you have advanced mode since current duplicate searching is already slow and I imagine this would be 10 times worse.

Alternatively a double search like "files where one file has these tags and the other has these tags" would be enough and allow you to, given enough searches, achieve a similar result.

Second: Being able to just see current tags or petitioned tags on file management. I often have similar files where the only difference is censorship, so I just copy the tags, but hunting for all the censor tags is a pain in the ass as is.

Maybe being able to search through current tags, but I can't imagine where else you'd use that function.


3718ce  No.12919

Does hydrus dev check this thread?


3718ce  No.12920

File: a2755c52c5c2aec⋯.png (98.19 KB, 456x526, 228:263, recent-searchs.png)

recent tag search


b63e40  No.12931

More awareness of exif/xmp and other embedded metadata.

For example, dupe checker being able to tell that two jpgs are 100% identical when metadata is stripped. Maybe even automatically mark as "same quality".

I've played around a bit with exiftool,

( https://www.sno.phy.queensu.ca/~phil/exiftool/ )

And a lot of photoshop jpgs have metadata about the original name it was saved under and even ID numbers pointing to the source image if it's an edit.




File: f16dc8fe807ba8f⋯.jpg (1.46 MB, 1289x1821, 1289:1821, 68390508_p0.jpg)

6f0155  No.12677[Reply]

Hydrus dev, I'm pretty sure I invented the proof to the big bang. If I get popular, I'll shout you out.

That's not why I made this thread, though. I'm so anxious I feel like vomiting. I never said this out loud anywhere, but besides porn I have nothing else in my life besides Hydrus. I use Hydrus extremely casually, anyway. But Hydrus helps sort porn, so… I was gonna shout it out.

I was gonna shout out my two favorite artists, too. I didn't have any plans to shout out anything else… So I imagine it'd be a huge influx of traffic.

I can't handle this at all. I never asked for this at all. No one will believe me until it happens anyway.

I have an appointment to show it to someone next week. It's fucked. I wish I could just be in protective custody and sleep for 20 hours and it'll be better then maybe.

8 posts and 1 image reply omitted. Click reply to view.

4a4bdd  No.12791

>>12719

The majority of people view the weak with disgust, that's the hard truth. It's kind of cruel that society pretends otherwise.


beb014  No.12842

>>12678

>>12677

>>12719

Sorry, I got caught up in the duplicates work last week and haven't checked the catalog for new threads in a bit. Assuming you aren't giggle-posting, it sounds like you are having a truly hard time. I sympathise deeply, but I am afraid I do not believe that happiness or self-esteem comes from getting love from others. I love you as a fellow Anon, but I am skeptical that repeating that sentiment would validate you.

The only one who can make your brain work right is you. As long as you believe you need affirmation from others to feel correct, you will be open to further abuse.

I strongly recommend you start getting exercise, if you do not already. Go for short walks every day. If you can scrape together a few bucks, get these books:

https://en.wikipedia.org/wiki/The_Feeling_Good_Handbook

https://en.wikipedia.org/wiki/Meditations

They are on Amazon and also in the library. If they ask, just say you are getting Feeling Good for a friend.


a3d3c4  No.12851

File: 7d72a28138cfb76⋯.png (1003 B, 113x60, 113:60, ClipboardImage.png)

wow very cool


c6ee8a  No.12860

File: c9e6a1d4848b3e6⋯.jpg (49.67 KB, 392x588, 2:3, cover.jpg)


b1b106  No.12912

File: 24fdbb89515d447⋯.jpg (72.41 KB, 1200x910, 120:91, Ck6tWhXUoAAUHNI.jpg)

>>12678

Stop lying about being the dev. Stop lying to yourself to achieve social results you don't actually want. Stop lying in saying that you didn't mean to say anything. Stop lying because if someone gave you what you asked for, you'd push it away because you didn't trust it.

Don't lie. Ever.




File: 5734ceedfcc53a4⋯.png (219.04 KB, 373x323, 373:323, Tzs0oi5.png)

211bff  No.10443[Reply]

Make my day showing me your boners, lads.

I hope i'm not a degenerate by this board's standards.

18 posts and 17 image replies omitted. Click reply to view.

423a17  No.12874

>>12872

How do you maintain privacy?


38288a  No.12879

>>12874

You dont. You just hope that the techno-jew doesnt care about whatever it is you're hoarding.


0d6500  No.12892

>>12872

>60 bucks per year

It's not expensive, but it's still 60 dollars more than my current backup expenses.


211bff  No.12898

>>12874

I don't give a fuck that some NSA employee can look at my patreon content.

>>12872

I like knowing that all my kino tiddies are saved somewhere in Buttfuc, Tennessee.

It's 3h of my job for a year of peace of mind.


0d6500  No.12908

File: 57e2c6b71dc3d74⋯.png (225.39 KB, 376x394, 188:197, ClipboardImage.png)

Finally reached 7% in regards to filesize by sorting the bigger files first.

There's still shit to import, but freeing 20GB up is pretty nice.




YouTube embed. Click thumbnail to play.

e99f2b  No.12755[Reply]

windows

zip: https://github.com/hydrusnetwork/hydrus/releases/download/v354/Hydrus.Network.354.-.Windows.-.Extract.only.zip

exe: https://github.com/hydrusnetwork/hydrus/releases/download/v354/Hydrus.Network.354.-.Windows.-.Installer.exe

os x

app: https://github.com/hydrusnetwork/hydrus/releases/download/v354/Hydrus.Network.354.-.OS.X.-.App.dmg

linux

tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v354/Hydrus.Network.354.-.Linux.-.Executable.tar.gz

source

tar.gz: https://github.com/hydrusnetwork/hydrus/archive/v354.tar.gz

I had a great week. The first duplicates storage update is done, and I got some neat misc fixes in as well.

false positives and alternates

The first version of the duplicates system did not store 'false positive' and 'alternates' relationships very efficiently. Furthermore, it was not until we really used it in real scenarios that we found the way we wanted to logically apply these states was also not being served well. This changes this week!

So, 'false positive' (up until recently called 'not dupes') and 'alternates' (which are 'these files are related but not duplicates', and sitting in a holding pattern for a future big job that wiPost too long. Click here to view the full text.

32 posts and 6 image replies omitted. Click reply to view.

e99f2b  No.12838

>>12831

Sure, let's give it a go. 64GB/week. You can see it in your review services if you refresh account right now. If it goes funky for some odd bandwidth-cycling reason, I can always reset it.


a07a0a  No.12841

Ok hdev, still going though video files for removal, and this is the 8th fucking video that is either the same or damn near the same as the others.

So this got me thinking on if I had the ability, how would I check duplicates for video. while i'm not coming up with good answers, I do have ideas.

first, could take an image every second, or so, and use them to fuzzy match, and if something fuzzy matches it goes a bit more process intensive to see if it is a real match. This would allow for both exact duplicates, and clips from larger files to be found. given what the dup finder can do with literal garbage, I think this could work if its even an option in the first place. depending on how big files are it would pull, it could bloat things out, but ultimately I think it would be an overall good.


e99f2b  No.12846

>>12841

Yeah, I do plan to add videos to the duplicate system, and I originally designed it to eventually support them. The recent file maintenance system was a step forward in prepping for the CPU work we'll need to do to retroactively crunch the data on this in a reasonable way.

I plan to do something like what you propose. The duplicates system currently works on comparing still images' shapes with each other, and it allows for multiple still image 'phashes' per file, so my task is selecting a good number of useful frames from videos that will match with others.

If it is reasonably possible, I would like do something more clever than just picking one frame per x time units or frames. This would line up right if our two vids were very exact conversions or resizes, but some of the codec changes drop a frame at the start or do 29.97fps vs 30fps bullshit that would desync our comparison. My original duplicates system did add vids by using the first frame, but so many have a black/flat colour first frame that it lead to a billion false positive dupes. Vids are no longer included, and I also drop anything that looks too much like a flat colour image from the system entirely. If I could instead find the x most 'interesting' frames of a video, then 2-second gif clips of 20-second webms would have a higher chance of being matched, and 30/60fps conversions would too.

I don't know, though. That is probably beyond me to do well, or maybe I can hack something that is good enough. I could do something like generating a phash for every frame in the vid and then have them compete with each other to remove similar-looking frames/phashes until the 20 most unique were left. It might pick up a bunch of false positives again with, say, a black screen with a bit of white text on (like an 'intro' title card) though.

Still, I am almost ready to do this now, and dupe work is proceeding, including more efficient storage of potential dupes, so maybe the answer here is to get a simple system in and then iterate on it.


a07a0a  No.12850

>>12846

when I say literal garbage, I mean the image is drastically different, to the point I cant even see how it thought they were dups, but those are from the asshole who specifically fucked with dup detection when creating trash images, the dup detector is able to have enough wiggle room that even if the images aren't lining up perfectly, it may spit out something useable, and I used 1 second just because no mater what I watch 1 second isnt enough time for a 100% scene change, it should pick up some duplicates from that.

on the title card, you could make a generic here is a black frame with text, and have a few variations of it, this could be used as a compare to X image and trace it if it does, so it would automatically know that everything with it will bee seen as a duplicate.

If you are able to, try to get in contact with the people from what anime is this, and see how they did theirs, it may give some ideas.


835966  No.12899

>>12846

Could always check how Video Comparer works. It's the best video dupe finder software I've used.




File: 4b005254620b5bf⋯.webm (375.91 KB, 480x320, 3:2, 4b005254620b5bf1b99582f85….webm)

116461  No.12858[Reply]

I had a good week doing small work. The error-popup issue with collected thumbnails is fixed, and there are new shortcuts to better allow keyboard-only navigation of the manage tags dialog, an experimental jpeg quality estimator for the duplicate filter, a fun clipboard watcher for easy URL import, and some new options for file viewing statistics.

The release should be as normal tomorrow.



File: ed3f745dbd39b5d⋯.jpg (4.66 MB, 4000x2715, 800:543, shutterstock_89245327.jpg)

12f73e  No.4475[Reply]

How about a thread for discussing/creating/sharing parsing scripts?

I made one for md5 lookup on e621.net (actually I just modified Hydrus_dev's danbooru script). Let me know if I did anything wrong with it, I'm pretty clueless… but it seems to work fine.


[32, "e621 md5", 1, ["http://e621.net/post/show", 0, 1, 1, "md5", {}, [[30, 1, ["we got sent back to main gallery page -- title test", 8, [27, 1, [[["head", {}, 0], ["title", {}, 0]], null]], [true, true, "Image List"]]], [30, 1, ["", 0, [27, 1, [[["li", {"class": "tag-type-general"}, null], ["a", {}, 1]], null]], ""]], [30, 1, ["", 0, [27, 1, [[["li", {"class": "tag-type-copyright"}, null], ["a", {}, 1]], null]], "series"]], [30, 1, ["", 0, [27, 1, [[["li", {"class": "tag-type-artist"}, null], ["a", {}, 1]], null]], "creator"]], [30, 1, ["", 0, [27, 1, [[["li", {"class": "tag-type-character"}, null], ["a", {}, 1]], null]], "character"]], [30, 1, ["", 0, [27, 1, [[["li", {"class": "tag-type-species"}, null], ["a", {}, 1]], null]], "species"]], [30, 1, ["we got sent back to main gallery page -- page links exist", 8, [27, 1, [[["div", {}, null]], "class"]], [true, true, "pagination"]]]]]]

48 posts and 16 image replies omitted. Click reply to view.

601c51  No.11814

I'm trying to use the iqdb-tagger python script, but there is a PermissionError when it tries to write to windows temp folder. Anyone know how to fix? I tried setting the iqdb-tagger-server.exe, iqdb-tagger.exe and python.exe to run as administrator but it doesn't help. I'm on Windows 10.

https://github.com/rachmadaniHaryono/iqdb_tagger


9d5f02  No.11886


e6a169  No.12763

>>7394

I've been using the tag parser and server (https://github.com/JetBoom/boorutagparser) fine until recently: random place he decided to host the sound went down, breaking a lot of shit. Thought I'd leave a note for anyone having problems: Just right-click on the script to edit it, then comment out (//) anything to do with the sound or variable it's stored in. That should get it working again.


bb00d3  No.12848

>>12763

I only use the parser, and just deleted the link to the audio file itself. Everything still works in the parser even with it there, but you get that stupid login prompt. And here I thought the boorus got hit with some new malware or something


bb00d3  No.12849

What's the deal with it not working on derpibooru anymore?




YouTube embed. Click thumbnail to play.

b60d7e  No.12669[Reply]

windows

zip: https://github.com/hydrusnetwork/hydrus/releases/download/v353/Hydrus.Network.353.-.Windows.-.Extract.only.zip

exe: https://github.com/hydrusnetwork/hydrus/releases/download/v353/Hydrus.Network.353.-.Windows.-.Installer.exe

os x

app: https://github.com/hydrusnetwork/hydrus/releases/download/v353/Hydrus.Network.353.-.OS.X.-.App.dmg

linux

tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v353/Hydrus.Network.353.-.Linux.-.Executable.tar.gz

source

tar.gz: https://github.com/hydrusnetwork/hydrus/archive/v353.tar.gz

I had a great week. I finished the basics of the new file maintenance system I wanted, cleaned up the duplicate filter a little more, and fixed a bunch of bugs.

file maintenance system

There are a number of large file re-checking jobs the client wants to do, both now and in the future. Going back to figure out more accurate video durations and image rotations, discovering webms that were formerly incorrectly detected as mkvs, eventually integrating videos into the duplicate checking system, all of these will require a combined whack of maintenance CPU that I don't want to hit all at once. I have previously sketched out some disparate systems for these jobs, but none were really doing the trick, Post too long. Click here to view the full text.

10 posts and 3 image replies omitted. Click reply to view.

b60d7e  No.12713

>>12683

Ok, thanks. I'll give 'move left/right one' and 'move to first/last' a go, which gives you four options, and we'll see how that works out.


7e46a7  No.12720

>>12710

Contrary to your opinion, and in agreement with previous anon, you are indeed a saint.


245cc4  No.12747

>>12713

Move left, move right, move first, move last

on the video tab I have

stupidly large remove

convert down

not hydrus

view again and decide

with

check before remove

music video

in for good measure

the first set is a wide range of reasons for a quick cull, the second set a second pass, with move left and right, I could easily set it up so its in between 2 tabs and move it to a second for more… hell, even a worst case scenario I could have multiple page of pages and filter that way.

not as set and forget as proper defined hotkeys, but should greatly increase the rate of sorting because I don't have to move them with the mouse anymore. definitely will be a god send once i'm out of video land.


245cc4  No.12753

Ill post this again in the new thread too,

I decided that the converted down images will get will get a rating of

Converted Down/Keep Large with 2 stars and not selectable as an option

it allows files to either be converted down and hidden from a search, or specifically keep the large file and hide it, also removed all files from the archive, everything needs a filter pass. these images that were converted down were interesting enough for me to keep/convert down but not really sure if I will long term storage them or not.

when I en mass rated them to converted down I got that error

InterfaceError

Error binding parameter 0 - probably unsupported type.

Traceback (most recent call last):

File "include\HydrusDB.py", line 561, in _ProcessJob

result = self._Write( action, *args, **kwargs )

File "include\ClientDB.py", line 12905, in _Write

elif action == 'save_options': self._SaveOptions( *args, **kwargs )

File "include\ClientDB.py", line 10112, in _SaveOptions

self._c.execute( 'UPDATE options SET options = ?;', ( options, ) )

sqlite3.InterfaceError: Error binding parameter 0 - probably unsupported type.

It doesn't seem to have made anything not work, everything's rated, so not sure what it is.


b60d7e  No.12847

>>12753

Thank you for this report. This is an odd error–the problem here is that the options object is not being serialised into the db correctly. This is the kind of error I see when someone has like a 2GB subscription, and SQLite falls over trying to make a buffer big enough for it. The options object there is a small thing, automatically saved at various points. Perhaps it was being nullified in some way, or some invalid data was being added to it. I can't think immediately why it would be affected by a rating set event.

Can you say more about this rating conversion? You had a 2-star rating service as set up as under manage services, and then when you did a giant ctrl+a->f4->set rating->ok, it took a moment to write that and then popped up this error right at that time?

Had you set any other options recently, either in file->options or via one of the 'cog' menu buttons? Anything related to default sort based on that new(?) rating service?




YouTube embed. Click thumbnail to play.

8e6284  No.12651[Reply]

ITT we post about our duplicates, tagging, and so on - our stats, goals, landmarks, and so on.

Never give up anons, that tidy catalog of content is waiting for you!

10 posts and 1 image reply omitted. Click reply to view.

8e6284  No.12724

>>12717

Ah yes, I also enjoy having a lot of useful files myself and having other people do the work, indeed.


82df6e  No.12758

anyone else cherry pick images then tag them individually as you store them in hydrus?

seems like if you mass download images you havent seen yet, you might have a bunch of images you dont care about


5ec0f2  No.12760

>>12758

I do both.

>you might have a bunch of images you dont care about

That's what the archive/delete filter is for.


ce936c  No.12765

I have 1TB of inbox to go through while 800GB are now properly archived, as in, I have personally checked them and stuff.

I've stopped working on dupes till the current dupe rework is done, since alternate not copying tags nukes my workflow, sitting at 80k dupes.


8e6284  No.12819

OP here, down under 60k dupes finally.

>>12758

I save those to Downloads with the most important tags in their title for later import.




File: ebcd0b5b00cd90d⋯.gif (1.02 MB, 1280x720, 16:9, ebcd0b5b00cd90d8f8fb165169….gif)

51a7cc  No.12749[Reply]

I had a great week. The new 'false positive' and 'alternates' duplicates db storage design and migration worked out well, although it did take most of my time. Other than that, I did a variety of little new options and bug fixes. The annoying bug where a handful of thumbnails sometimes stop fading in seems to be finally fixed!

The release should be as normal tomorrow.

f7545c  No.12751

I like how op starts off with how well he's been doing


eb7ef1  No.12752

>>12751

I love hydrus man and hope he is doing well in all aspects of his life.


51a7cc  No.12754

>>12752

>>12751

I am overall healthy and in a stable money situation. I hope you are as well. I get blackpilled sometimes, but then I remember my true troubles are all internal and that I have a lot to be grateful for compared to my ancestors. I still want to keep pushing on Hydrus every week for the foreseeable future. I love imageboards and all other Anons, including you–the cheeky fun we get up to keeps my soul going.


a84085  No.12762

>>12754

Based and Bloomerpilled




File: 4d6f470068e6acc⋯.jpg (338.2 KB, 1925x1080, 385:216, 1517368870.jpg)

da8629  No.12692[Reply]

To quit spamming the Q&A thread (sorry) I agree this should be made into its own thread.

I'd like this to be a place to ask questions on "how should I tag this" for those that don't use the PTR and think you may be able to be more intelligent with how you tag. Also a place to discuss how you tag things and, most important, WHY that way. For example, what sort of issue were you looking to solve or what did it make easier by doing it that way?

T

o start off the discussion - I'm going to write a small essay on my thoughts on tags/tagging. It is my belief that tagging should be, above all else, two things

1) Simple - There are few scenarios where you must think if something should be tagged or not. Edge cases and oddities should be minimized at every opportunity. The number of tags should be kept small enough to maintain the entire list in memory. This is important for consistency. Speaking of which, the second item is…

2) Consistent - Tags should be well defined and used consistently. If a tag is often forgotten to be applied then it is a bad tag and should be either removed from the system or renamed/redefined as to be applied more consistently. In order to keep consistent tags it is also a requirement that all tags are able to be kept in memory to be applied properly. If you have so many tags that you are incapable of remembering them all then you will likely be tagging inconsistently. It is okay to add new tags to cover areas.

A tag that is used infrequently because the number of items in the gallery are too few is acceptable but is discouraged if the term is rarely searched for (either as an inclusive or exclusive term). Remember: tags are meant to help you search for images. This is important when it comes to naming tags.

I use a collection of 288 unnamespaced tags and 11 namespaces. The tags are mostly descriptors of clothes, physical traits, and objects. Namespaces are used for more meta information like character: artist: series: or whether a piece of art is SFW or pornographic.

Some of the more heaPost too long. Click here to view the full text.

4 posts and 2 image replies omitted. Click reply to view.

61ce05  No.12714

I try to be practical about it and take a few things into account:

>This is private, I'm doing this myself

That means I use tags based on how I would search for an image later. I'm not doing an essay in tags for every minor detail I will never remember. The main purpose for me is to find images so I tag in a way that I will find them.

>This is private, no one but me will ever use this set of tags

Ties in to the first but it means very personal tags. You woudn't use "cute" on a booru, for example.

>I have a fuckton of images to sort and there's always more to come

That's why I keep it down. There's rarely more than a handful of tags and if there are, it's obvious things. A knight for instance will have something like "knight, armor, helmet, sword, shield, fighting, blood, deus vult" and there might be more because I utterly made that up.

I do however use tags for one other purpose other than finding things: to provide source. I do try to be specific with the creator: series: and character: tags for that reason.

Apart from that, since I see no reason to run an SFW database right now, I do have rating tags for questionable, explicit (this includes non-sexual stuff too, like a certain australian's lifestream) and pornographic, the latter two being a parent-child pair. So in the rare case someone is standing behind me I exclude those.

Oh, and I tag if it's a gif, a video, a readable or an audiofile.

One thing to remember when tagging is that you don't need nail the tags down so exactly from memory that you only find a single image. There's a nice big screen full of thumbnails. I firmly believe that some visual searching is far faster than trying to go by tags alone.

Pro tip: If you want to replace a tag entirely, not just via sibling, fetch all files with the old one, tag them new and remove the old one. I did this a few times already.


da8629  No.12716

File: f0e8cedb906717e⋯.png (992.75 KB, 1000x1414, 500:707, f0e8cedb906717e7356534dbe9….png)

>>12706

Yea, I figured that was the case. It's quite pragmatic but god does it result in so many bad and redundant tags…

>>12714

I agree about the private use meaning you can use tags that have personal meaning to you. "Cute" is a useless tag when tagging for other people but is very useful on a private collection to tag things you think are cute. I'd actually be interesting in hearing the kind of personal/private use tags people use as sometimes they're good ideas for the kinds of things that are worth tagging that I may not have thought about. I have a tag "waifu_material" for example.

>One thing to remember when tagging is that you don't need nail the tags down so exactly from memory that you only find a single image.

Not sure if you're saying this as a general thing or a response as a misunderstanding to what I said about memorizing tags.

I don't think you should know what every image is tagged from memory - but rather that you should be able to keep all existing tags available to use in memory. The creation of new tags (that aren't namespaced like series: or character:) should be extremely rare in a well planned system. Forgetting to use a new tag is OK for a period of time but if you're consistently forgetting to use it then it is simply a bad tag.

The worst tag is a tag that shouldn't be there but is. The second worst tag is one that should be there but isn't.


aae946  No.12729

>>12705

>I currently don't use the rating system and can't even figure out where it's hidden under.

Services > Manage Services > Add > Local Numerical Rating Service

The beauty of rating services is that they will not be carelessly pushed to the public repo, which is great because I think explicitness is very subjective


0a9d65  No.12741

> Metholodogy

Why not we gather all the major ways of managing Hydrus and create a "common workflow"?


9d3e84  No.12748

File: 76ab2e2ef12fa4e⋯.swf (8.56 MB, 76ab2e2ef12fa4e51dbe4d865a….swf)

I'm not sure if this goes here or in the questions thread, but how should I tag flash codes in the PRT?

Boogie usually puts scenes in his flashes that are only accessible by code. Flash related has two alternate scenes you can access by typing 'paper' and 'yokai'. I think those are useful enough that I should put them in the PTR, but I don't think anyone has done anything similar, so I'm not sure what namespace to use. Maybe 'code:' or 'cheat'?




File: 48f7d79317b8899⋯.gif (381.04 KB, 350x188, 175:94, 48f7d79317b88998c4e3dc1218….gif)

673973  No.12665[Reply]

I had a great week. I finished the first version of the new file maintenance system I had planned, cleaned up the duplicate filter workflow a little more, and fixed a bunch of bugs.

The release should be as normal tomorrow.



YouTube embed. Click thumbnail to play.

e9f34f  No.12583[Reply]

windows

zip: https://github.com/hydrusnetwork/hydrus/releases/download/v352/Hydrus.Network.352.-.Windows.-.Extract.only.zip

exe: https://github.com/hydrusnetwork/hydrus/releases/download/v352/Hydrus.Network.352.-.Windows.-.Installer.exe

os x

app: https://github.com/hydrusnetwork/hydrus/releases/download/v352/Hydrus.Network.352.-.OS.X.-.App.dmg

linux

tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v352/Hydrus.Network.352.-.Linux.-.Executable.tar.gz

source

tar.gz: https://github.com/hydrusnetwork/hydrus/archive/v352.tar.gz

I had a good week. .ico files are now supported, 'collect by' status is remembered in gui sessions, and I fixed a bunch of bugs.

duplicate overhaul plans

I started the duplicate overhaul work this week with some planning and experimentation with existing data. My original thought here had been to exactly replicate existing functionality just with a more efficient database schema, but having gone through the various edge-case insertion and merge operations, I believe the current system is overcomplicated for what we are actually using it for.

Most of all, the current system tries to form a chain of 'better/worse' comparisons so all dupePost too long. Click here to view the full text.

19 posts omitted. Click reply to view.

e9f34f  No.12637

>>12630

I assume you have already done archive/delete filter on these files, so they are all in the archive? Otherwise I would suggest using inbox and archive to track files you have processed or not. You can send files back to the inbox for another round of archive/delete processing with Shift+F7, btw.

I have a couple of like/dislike ratings on my client that I use as pseudo inboxes. One is called 'read later' that I apply to anything I am archive/delete filtering that is like a big thread screencap or a 15 min video on some bullshit that I don't want to deal with in a quick filter. I enjoy like/dislike ratings since I can just click to quick apply and have some shortcuts set up for this as well. Maybe you could make a new like/dislike rating called 'videos to process' or something with some shortcuts to like/nullify to help you keep track of all this?

Mapping things to tabs is tricky. I'll be doing some Client API tab-access work soon, so I'll be thinking more closely about this in a couple weeks. The tricky part is mapping the action to the tab–'send file right one tab' is fairly easy to manage, and has a simple yes/no error state, but 'send file to page named "sexy elves"' is just that step more difficult to track and action and fail with (e.g. there could be two pages with that name). I like the idea of setting a current 'destination' tab in the current session that you could then action. I'll think about that. I've thought maybe having a shortcut or action for 'suck up these thumbs into the mouse' and then 'spit those thumbs back in here', like in cut and paste terms.


7ccfe8  No.12648

>>12636

Oh, well that's good to hear. I don't have any problem with the PTR, I just thought you did since people talk a lot about tags in it and I remember there was a lot of complaining a year or two ago about wrong tags. I just use it as an aid to my own tags to sort stuff, so I'm happy if you're happy.

I do want to try running my own PTR and file server with Hydrus if I ever get the money and time, so if I do I'll give feedback.

As for that workflow idea, I'll give it a try with some files once I finish with my duplicates.


345431  No.12654

>>12633

I suggested 30 minutes as the obviously safe option, where no one could possibly watch a preview for that long, but the majority of AFK sessions would be longer than that. I wouldn't expect to see much reduction in sessions pruned by going with 10 minutes, but who the hell is legitimately watching the preview for 12 minutes and wants that time recorded?

Honestly, I'm not convinced this needs to be a setting. 10 minutes is way more than almost anyone will ever watch in a thumb. 60 minutes would cut out the vast majority of AFK sessions. It's such a broad range to work in that 30 minutes would do the right thing in 95%+ of cases (and I'm being cautious with my numbers), so why add more clutter to the settings for something almost no one would need to change?


5a219e  No.12656

File: 1ecb47d56ca76c4⋯.jpg (8.4 KB, 815x24, 815:24, media viewer.jpg)

>>12633

>>12654

Could any potential solution for this be extended to the media viewer? This seems like the same problem of getting distracted during, say, an archive/delete session, and leaving one image open (but minimized) for a few hours.

I've had this, but also a similar problem with the media viewer (pic related). I basically had a bunch of 'alternate/duplicate' files, and I was flipping between all of them hundreds of times trying to decide which to keep, tags etc.

Is it possible to detect if the client doesn't have focus, and stop tracking immediately? I feel like that would solve 90% of issues right off the bat. Otherwise, tying it in with the client's inbuilt 'idle' state could also work well (and is already user configurable in options -> maintenance and processing).


e9f34f  No.12845

>>12656

>>12654

Thanks lads, only catching up now. This isn't in yet, but I'll keep this in mind as I do this system.

I'll do min/max for media viewer as well. Options are easy to add, and I know someone will want to say 'no minimum time' or whatever, so I'll throw them in for anyone who is interested. 5s/10m sounds like an ok default min/max.

I think I can catch a focus lost event, although some of that stuff is a little unreliable. I'll play around with it and we'll try iterating on this a bit.




Delete Post [ ]
[]
Previous [1] [2] [3] [4] [5] [6] [7] [8] [9] [10] [11] [12] [13] [14] [15]
| Catalog | Nerve Center | Cancer
[ / / / / / / / / / / / / / ] [ dir / 2hu / asmr / choroy / dempart / general / leftyb / mde / vichan ]