Manga Crawler Wishlist

Jul 28, 2010 at 6:13 PM

Hi AnuTom,

at first thank you very much for your software, it's great :) I'm glad that somebody is developing an alternative to DomDomSoft Manga Download (http://domdomsoft.com/home/), that is the software I used until today for my manga download needs, so I hope that you will keep updating it :)

I tested MC by the DDSMD user point of view, so I'd like to share with you some opinions.

MC pro:

- it's fast! and it loads manga lists on demand and automatically, not manually and sometimes like on DDSMD;

- it lists tons of nice manga sites and some of these (like 1000manga) are updated more often than those ones on DDSMD! Example: last chapter of "Sidooh" listed in DDSMD is # 56, while in MC is #66.. whoa!

 

MC cons / wishlist:

- MC misses a "zip chapters folders + rename it to cbz" feature, so i have to do it manually (DDSMD contains this feature);

- MC misses the ability to set the number of concurrent downloads;

- a primitive way to bookmarks manga could be a nice add in ;)

 

I hope this post will be useful to you! Please keep coding this cool app :)

-numbw

Coordinator
Jul 31, 2010 at 12:36 AM
Edited Jul 31, 2010 at 1:00 PM

- MC always show updated chapters list, because it parsing html pages

- cbz, bookmarks - i will look at this

- concurrent downloads - but only from diffrent hosts

Aug 4, 2010 at 6:48 PM
thx :)
Nov 5, 2010 at 12:47 PM

this is a great job.

but i'm still a user of domdomsoft manga downloader because it has spanish sites as well.

if is not much trouble, can you add at least:

http://animextremist.com/mangas.htm

http://submanga.com/series

áctually, don't rename zipped files to cbr

i agree that a bookmarks or other feature for remember the downloaded mangas will be usefull.

thanks again for your work.

May 30, 2012 at 5:13 PM

With the additions of the bookmarks and chapter compression the tool became really awesome and my favourite!

 

It wood be great to add an option to delete the chapter folder after the chapter is downloaded and compressed (of course only if compressed)

 

And it seems i'm having a little trouble with the zipped chapters.

The MangaReader under Win7 shows the pages ordered as 1,10,11,12,13,2,3,4,5,... not in the correct order. (On my Androidtablet, it is working correct ...)

It would be great to add an additional page naming strategy like "trim to three digits" (images will be named like 001,002,034,055,234,...).

 

Did those chances locally for me, and if you're short on time i can upload the changes into a fork.

Coordinator
May 30, 2012 at 7:18 PM

If you have done that already so please make a fork. I'll look at those changes.

Jun 11, 2012 at 11:58 AM

have also added a crawler for MangaStream on my fork

Coordinator
Jun 11, 2012 at 9:40 PM
thanks for your contributions
shame to say, I was unaware of your previous fork commit, and I written myself requested by you features. But I have reviewed your changes and they are quite similar. So I plan to keep mine, with some code from your.
Now, that you not think I don't read your code:
[YAXNode("DeleteChapterFolderAfterCompress")] - need something like this [YAXNode("DeleteDirWithImagesWhenCBZ", Default = false)]
so old xml will read. Didn't you get any exception with new code ?
Instead new naming strategy I have added new checkbox which allow padding with zeros.
And numbers of zeros always based on pages count.
Your code:
if (Index > 99)
return Index.ToString();
else if (Index > 9)
return String.Format("0{0}", Index);
else
return String.Format("00{0}", Index);
Mine:
if (DownloadManager.Instance.MangaSettings.PadPageNamesWithZeros)
index = index.PadLeft(a_pages.ToString().Length, '0');
When there are lets say 500 pages your code just generate: 001,02...009,010,011,...,099,100,
When there are 99 pages you generate: 001,02...009,010,011,...,099
Instead my code in latter case generate 01, 02, 11, 99
Still I don't know if I really should only pad or pad always to 000. And what when there are 1200 pages ?
And about clearing folder after zipping.
Your code:
// delete zipped chapter
try
{
if (DownloadManager.Instance.MangaSettings.DeleteChapterFolderAfterCompress)
if (State != ChapterState.Error)
dir.Delete(true);
}
catch (Exception ex)
{
Loggers.MangaCrawler.Error(String.Format(
"Exception, chapter: {0} state: {1}", this, State), ex);
}
I strongly encourage to not delete whole folder without moving it to recycle bin. You need to know how stupid users can be. Instead we should delete all our images and delete folder only when its empty. And why not puts this code in upper try catch block
As for Manga Stream crawler, do you plan to make some tests :) in MangaCrawlerTest ?
I know this is tedious work but important one. If so try to test some finished chapters and some ongoings, there are often some diffrences when you click on last image of series, if you can click it at all.
Second, try to run _RandomTestAll, remove all other crawlers except your, keep it for few hours, analyze errors (stop test, see test results),
and so on as you will be sure you can handle most on strange and stupid things.
Try to look for licensed series, adult series, series with volumes (one more level in tree structure), try if it's possible to remove empty series,
Surprisingly this tool should be used by owners of services to find errors, chapters without images, etc.
By the way how you get xpath from html, what tool you are using for it.
And last, but this will be my job, catalog needs some rewriting to be ready for new crawlers.
Well I have looked on manga stream www and its really simple, nonenthles some test are required.
Currently I plan to integrate your work, check all services, and repair broken one and release maintance release
Jun 11, 2012 at 10:38 PM

Hehe, no problem at all, my fault for forgetting to sent a pull request :)

Also i have no problems if you keep your code! I only suggested this because a friend of mine thougt it would be nice to share the addidtional features to more people. I'm still counting me as a beginner in c#, so i was mentally prepared for this :P

No Exceptions/Problems with the YAXNodes on my side.

About the padding, it was faster and easier for me to implement the simple three digit method, of course it is better to pad to the number of pages (though i doubt there are chapters with over 999 pages xD)

The catalogue probably really needs a rewrite, when a new crawler is added bookmarks and downloaded chapters are smashed.

 

For my xpath queryies, i used the HAP Explorer that is offered on the HTMLAgilityPack site (http://htmlagilitypack.codeplex.com/releases/view/44954)

Probably not the best one, but it worked for me (complex queries often freeze the tool for some seconds)

 

Oh, the newly added MangaEdenCrawler (english only, no adult), still has at least one bug, (first chapter of a series will fail downloading. Was a bit too quick with pushing to rep) i'll try the testing unit on it after i have taken some hours of sleep.

The ME was probably the last Crawler addition from me and with it this tool has all sites i need :)

Aug 22, 2012 at 8:30 AM
Edited Aug 22, 2012 at 8:33 AM

Would be great if you can add support for

+ FoOlSlide (http://foolrulez.github.com/FoOlSlide/ example: http://foolrulez.org/slide/reader/)

and older version

+ FoOlReader (https://github.com/woxxy/FoOlReader example: http://sakicow.com/reader/)

Then users would add pages based on them to Manga Crawler be hand.

More FoOl* sites: http://slide.extrascans.net/ , http://manga.redhawkscans.com/reader/ , http://reader.imperialscans.com/reader/ etc.

+ Batoto

 

* Check if there is newer version of Manga Crawler

* Option to hide unwanted servers from list

 

Best regards

Feb 21, 2013 at 5:46 PM
Hi, firstly - great app, but I'm writing to ask about that zero padding. Will it be implemented soon? It'd be extremely useful option, even with just 3 digits (pages and chapters), especially for e-books readers converting.
May 12, 2013 at 2:34 PM
Hey tomanu, this app rocks me!!! in addition, i would like to suggest some feature :
  • log and actual image-url viewer so that in case error occured user can manage to fix it themselfs
  • download option- i believe this apps functionality is to download manga, right?? so why don't you include option that exist on download manager like IDM, that would be great!!!!
  • can you port this app to c/c++ and make it independend from any framework??
Nov 2, 2014 at 3:15 PM
Cool app ,but there are a lot of errors.
Animea ,Kissmanga ,Manga Fox ,Manga Here,Manga Reader ,Manga Volume and Starkana do not work ,could you fix them please :)
Nov 20, 2014 at 4:35 PM
Why does download newers before olders?
When i want to download a manga i want to star reading it from the beginning.
Jan 22, 2015 at 3:50 PM
plz add
http://bato.to/ to the list
Jul 19, 2015 at 3:58 AM
hope that the downloaded manga folder and rename it like naruto[001], naruto[002], and so on, I like to arrange my folder in that way.. :)