The Website Parser - Plugin Information and Downloads

Wir haben aktuell ein Problem mit dem Board und arbeiten an der Lösung...

    Sharkadder:
    First I like to thank you for your answers and also that you keep an open mind. When I start "criticising" something, I try to be positive and make suggestions. I'm happy you are willing to listen. Thanks for that.


    For the proxy stuff:
    In Python's urllib2 is proxy support already built in.
    http://docs.python.org/library…html#urllib2.ProxyHandler
    A quick check gave that it seems to be available on dreambox too, at least on 1.5 images original by DMM.
    That means if you use python's urllib, you would need to use urllib2 instead and just use the class urllib2.ProxyHandler.
    Pls. see:
    http://docs.python.org/release…lib/urllib2-examples.html


    In that case, all content fetching (including images) would occur on dreambox side and sent over configured proxy.
    Of course, the proxy needs to be configurable in your plugin thus it requires you to write additional user code.


    Another thing is to separate the real "remote http connect" in a try/catch/exception block - with or without proxies. Therefore you can implement an exception handler, which jumps out of application in a coordinated way rather then just crashing. I guess you are already using exceptions (have not seen your code yet).


    So before starting to fetch remote URLs, you could just test the connection - this is advisable for every networking connection anyway.
    Would help in temporary internet outage also :)


    For your user announcements and README infos: you could implement that on a per-demand basis, just map it on HELP oder YELLOW button :winking_face: So if somebody wants to know the latest NEWS about the plugin, he hits the $MAGIG$ button ...


    Oh and no, I'm no dreambox programmer but did a lot of coding for the old XBOX and the XBMC project which is also python based (wrote an email imap/imaps mail client with print functions, attachment views and the like for xbmc).


    Hope I could help.
    _marv_

    Well thanks for the input, i did originally look into urllib2 but when i tried it my 8000 wouldn't handle it, i later got told it was not supported at the time when i first developed the plugin. Another thing is, it doesn't really matter how i program, i'll always get somebody with a strange box model complaining that it doesn't work properly, satellite boxes are very tempremental and have different effects on different makes/models or different images.


    I know that HTTPparser is now available to use on the box, this function was also not availble when i first developed the plugin. So you think if i use urllib2 i should be able to allow proxies? If urllib2 does now work on the box then i shall include this.


    Another thing too, i do only use try/catch blocks once images download for download/failed download, i am not sure why because in another language i use called C# i use them all of the time, i think my code needs a good makeover i think rather than just trying to cram as many functions in as possible...that is probably why people are getting impressed with the work rate, my code admittingly is very messy at the moment! I'll see how this next version goes but seriously i am looking to embark on new projects soon rather than website parsing ones.

    Well, that explains it a bit.


    I tried following code:


    #######################
    import urllib2
    proxy_handler = urllib2.ProxyHandler({'http': 'http://www.hidden.org:80/'})


    #commented out since this is a proxy without authentication
    #proxy_auth_handler = urllib2.HTTPBasicAuthHandler()
    #proxy_auth_handler.add_password('realm', 'host', 'username', 'password')
    #opener = build_opener(proxy_handler, proxy_auth_handler)


    opener = urllib2.build_opener(proxy_handler)


    # This time, rather than install the OpenerDirector, we use it directly:
    content = opener.open('http://www.google.com/')
    print content.read()
    #########################
    and it works on my DM8000 with opendreambox-1.5 (DMM image)


    Just try it on a ssh/telnet window...



    EDIT:
    there is a well commented man page for urllib2:
    http://www.voidspace.org.uk/python/articles/urllib2.shtml


    EDIT2:
    It seems that if urllib2 is used, it utilizes the http_proxy variable automatically. So our iranian friend would need to set these environment variables either within the plugin or even outside of enigma2, directly after system boot. In that case enigma2 would use this environ vars and so do the scripts started from within enigma2.

    ok thanks marv, i shall look into using it in my next version. I may need peopleto test it once done and so i'll post a reply once i've changed things around a bit . As i say, i only used the urllib because that was all what was available to me at the time i started to devleop this, i've just kept the same system since you see as it was easier for me, i never thought about these issues.


    p.s. what is the hostname, username and password all about for the handler?


    Mark

    Glad I could be of assistance.


    The hostname/user/pass values are meant for basic authentication.
    That is if a website requires authentication of users, they need to be specified there.
    But the vars can be used also as proxy authentication params as well - in case your proxy requires authentication.
    You'll find examples in the mentioned links.

    Hi,


    thx for the plugin.


    Is the "European satellite footbal channel listing" implemented now?

    The world satellite feeds? Yes it is dude along with one off feeds too :-), it also states what is available on the news page when you first load the plugin up and so all you need to do is refer to that for updates :-). The satellite feed images are a bit small at the moment and i'll try and make them bigger in the next version if i can.


    It seems i have now had a set back with my next version due to me needing to change a few things and so it looks like it will not be out for the start of the EPL season...I will do my best for this to remain a possibility though ;-).


    Oh and to marv, you should look at doing some enigma2 programming, it seems like you're more experienced with python than i am and a few people i know have ideas for some projects to undertake if you're interested :-).


    EDIT for 6462 and marv
    ===============


    I have spoken to a few guys about this whole "proxy" business and it turns out that the dreambox automatically detects proxy settings itself. As for this being implemented i had some advice to say it should be include by dreambox devs for use in all plugins, although since my plugin is one of the only ones to require such a thing then maybe i should implement it inside of mine instead. I have looked at urllib2 and yes it is more or less the same as urllib, however urllib2 does not have the urlretrieve function in order for me to copy a file from one destination to another and the shutil module does not allow for proxy settings and so i may be back to square one with that one.


    From what people have told me is, my plugin should be fine if you purchase a proxy from somewhere since i've been told that dreambox detects these settings automatically apparently. the following url has proxy servers in which are in the uk:


    http://www.xroxy.com/proxy-country-GB.htm


    if you type proxy settings into your router then it will enable you to view the website fine, how you exactly do that i am unsure but i'm sure you can read it up.


    p.s. sorry if i am not allowed to use that link, it is just a link what livescore creator gave me


    Well TTFN

    Hello,


    I have now released the new version of the website parser, in this version you will be able to:


    Display world satellite feeds for sport events
    Display one off satellite feeds for sport events
    Zap to a channel for a satellite fixtutre (if transponder is valid)
    Zap to a channel for a one off satellite feed (if channel reads and decodes on dreambox, this does not work for 4:2:2 feeds).
    Manually scan in channels if it is not already scanned in when wanting to zap
    Position your motorised dish if manual scan shows 0 results
    Football scores, results, fixtures and news
    FIFA RSS feed now included
    Vide Printer
    UK Music charts
    UK HMV charts
    UK Cinema listings


    The supplied screenshots show the new menu layout and also the steps i went through to channel zap to "Sly sports 2" in the UK. As you can see from the screenshots, it works well.


    Many thanks,


    p.s. Sorry for those who tried to update on the afternoon of the 16th November, i was doing some final testing and so the version on there was not final and would have made your box crash. Simply re-download version 0.1 and update, problem is now solved. If you did not try to update on that afternoon then just update as normal via yellow button...works great for me.


    p.s.s. version 0.1 is attached, simply update via yellow button to get V0.5 once V0.1 is installed


    Many thanks,


    Mark

    hi,


    I do not understand your request with "Can a sweet version spacing is greater?", maybe bad translation there.


    I assume that you mean can i make the spacing bigger. I'd of thought it would be more readable on your 42" as the text is bigger but i do not know. My 26" i test on is just a few feet away from me so i can obviously read ok, so i'll look into it.


    I hav been asked about making images bigger but they get pixelated as i increase the size. I can make them a bit bigger and will try to include something a bit more readable if i can in the next version.


    Thanks for the feedback.

    Hi,
    Please assign a section for transponder news it is very helpful to be aware of last channels frequency and transponder changes. Flysat.com and kingofsat.net are two famous websites for presenting latest Transponder updates especially flysat.com that updates hourly.
    Downloaded image for satellite feed seems to be small I can hardly read it. I don’t know Maybe need some sort of layout or changing the size of windows.
    Thank you Again

    Zitat

    Originally posted by 6462
    Hi,
    Please assign a section for transponder news it is very helpful to be aware of last channels frequency and transponder changes. Flysat.com and kingofsat.net are two famous websites for presenting latest Transponder updates especially flysat.com that updates hourly.
    Downloaded image for satellite feed seems to be small I can hardly read it. I don’t know Maybe need some sort of layout or changing the size of windows.
    Thank you Again


    Yep, as i mentioned in the previous post, i am aware of this problem about image size. As for the transponder updates, a mate of mine suggested including such a thing, i will look at that too ready for the next version or the version after that. At the moment i will be working on a search box for upcoming internet movie database in my plugin and then i shall include that. Expect another release before Christmas with new bigger feed pictures and also possibly transponder updates from flysat.com. If i can, i will try and replace old transponders on the dreambox with the new ones quoted on flysat or kingofsats but i will see how much progress i get done in the new few weeks.


    Many thanks for the feedback,


    Mark

    Hi
    I've installed the Version 0.1 on the GP3 but when I subsequently updated to the latest V .05 it crashed and had to delete it. Any help? Thks.

    Zitat

    Originally posted by choco
    Hi
    I've installed the Version 0.1 on the GP3 but when I subsequently updated to the latest V .05 it crashed and had to delete it. Any help? Thks.


    Zitat

    Originally posted by dreamboxrss
    After Update and Restart i have a greenscreen when i start the Plugin


    crashlog

    Code
    IOError: [Errno 2] No such file or directory: '/usr/uninstall/RemoveTheWebsiteParser.del'


    this plugin needs the directory /usr/uninstall/ and an empty file called RemoveTheWebsiteParser.del


    create empty file RemoveTheWebsiteParser.del upload it in /usr/uninstall/

    Same result as before:- V 0.1 ok. Then, as before, updated to v.05 and as soon as I started the Plugin, I got the green screen and had to delete the plugin.

    I think that is not the problem, the problem probably might be the following:


    Code
    AttributeError: 'NoneType' object has no attribute 'isTmp'
(PyObject_CallObject(<bound method ActionMap.action of <Components.ActionMap.ActionMap instance at 0x98cc88>>,('WizardActions', 'ok')) failed)


    But i don't know this error message. :frowning_face:

    Dear users,


    I have just noticed that many of you are now experiencing problems with my plugin on later images. Since gemini project 3 is an up to date image i have not been testing on it in the past, nor have i got people to test using it.


    Before i releasethe new version of my plugin i will make sure that it is compatible with this piece of firmware. The uninstall directory issue will be due to gemini not allowing it to be written to the directory, other than this i cannot see why such an error would arise but i will look into it. The delete/uninstall file should be there as it gets downloaded from my server on update, i will make sure this definately gets installed on the next update.


    If in doubt
    create empty file RemoveTheWebsiteParser.del upload it in /usr/uninstall/


    This advice is good for now, if you still have problems then wait for version 0.5a, it will be a few weeks away as i am currently working on a search engine for IMDB script.


    Thanks for all of the input, these errors and crash logs are much apprechiated, but the uninstall error is nothing new, some images seem to have problems writing it to the right directory.


    EDIT
    ===


    I have checked the crash logs in which you have all submitted, you are all experiencing the same error, i will as i say edit this in the new version to create a remove file from the box itself, rather than downloading one from my plugin to save such issues.


    Many thanks,


    Mark (TWP Creator)