Google appengine handling external services and xpath

Two really frustrating things that one needs to adjust to when writing python applications for google's appengine, which for me I hit straight away because I was writing a 'mashup' type webapp which involves consuming services and processing the data into something new and hopefully fanstastic


1. XPath'ing is not really part of any supported library

The closest way I found after many hours of frustrated search is to use the ElementTree library, which provides a very very rudimentary "xpath" type feature

from xml.etree import ElementTree

doc = ElementTree.fromstring(xml_str)
for element in doc.findall('something/id'):
  print element.text

2. Google appengine is very picky about timeouts when using urllib2, use urfetch instead and specify a deadline

Anything beyond I think about 4 seconds seems to cause exceptions, so you're better off to use urlfetch.fetch and specify a nice fat deadline.

from google.appengine.api import urlfetch

result = urlfetch.fetch(url=some_url, headers={'header': 'something'}, deadline=25)
if result.status_code == 200:
  return result.content
return None

However, i've noticed that it's still possible to get the following DownloadError: ApplicationError: 5 exception even when you're using a gracious deadline..

    result = urlfetch.fetch(url=url, headers={'Agent': 'yes'}, deadline=25)
  File "/base/python_runtime/python_lib/versions/1/google/appengine/api/urlfetch.py", line 241, in fetch
    return rpc.get_result()
  File "/base/python_runtime/python_lib/versions/1/google/appengine/api/apiproxy_stub_map.py", line 501, in get_result
    return self.__get_result_hook(self)
  File "/base/python_runtime/python_lib/versions/1/google/appengine/api/urlfetch.py", line 331, in _get_fetch_result
    raise DownloadError(str(err))
DownloadError: ApplicationError: 5 

and for that i have no idea what todo about :D

0
Your rating: None