Skip to content. | Skip to navigation

Personal tools
Log in
Sections
You are here: Home Posts Moving to Picasa Update

Moving to Picasa Update

by Nathan Van Gheem last modified Jul 11, 2011 02:08 AM
Everything didn't go as well as I hoped, but I fixed all the problems I was coming across...

While moving all my photos from flickr over to Picasa Web Albums I ran into some problems. The original post explained how the whole process works.

  • Picasa albums have 1000 photos limits
  • the python gdata api didn't support video uploads
  • Picasa trimmed whitespace from Album names
  • other small things
The new script includes creates a new InsertVideo method to support video uploads, moves multiple photos at a time, automatically creates multiple albums if there are more than 1000 photos in a set and fixes other small issues. You'll need to install the flickrapi, gdata and ThreadPool packages.
It completely works and is done now though. :)

Update

There is a new version of the script to handle image resizing. Customise the script to control the size to resize to.

Filed under:
Igmar says:
Jul 10, 2011 06:41 PM
Great work !
Is it possible to modify this script to resize the photos when bigger than 2048x2048 before uploading to Picasa because if you’ve signed up for Google+, photos up to 2048 x 2048 pixels and videos up to 15 minutes won’t count towards your free storage.
Thanks
Nathan Van Gheem says:
Jul 11, 2011 02:11 AM
Just updated the article with a new version that handles resizing. This is completely untested but *should* work. I don't have anything to test it out with any longer since I've already moved :)

Let me know if any more changes to the script are needed for it to work.

Also, PIL is required to be install in order for this part to work. If you don't have it installed, it should be rather easy to find the python package in any package manager.
Igmar says:
Jul 11, 2011 08:22 PM
Thanks a lot for your answer but I cannot download the migrateflickrtopicasanokeyresize.py file, log in is required :(
Nathan Van Gheem says:
Jul 11, 2011 09:50 PM
Sorry about that. Forgot to publish the file. Give it a try again.
Jack says:
Jul 12, 2011 02:46 AM
Hi, I keep getting this error - I'd love to find a way to fix it! Let me know what I can provide to help out.

gdata.photos.service.GooglePhotosException: (403, 'Forbidden', 'Not authorized to modify this resource.')
Nathan Van Gheem says:
Jul 12, 2011 03:00 AM
Can you provide all the output? Not just that exception.
Eugene Lazutkin says:
Jul 14, 2011 07:40 AM
I am getting following errors (one example of many):

Traceback (most recent call last):
  File "/usr/local/lib/python2.7/dist-packages/threadpool.py", line 156, in run
    result = request.callable(*request.args, **request.kwds)
  File "./migrate.py", line 255, in move_photo
    im.save(filename)
  File "/usr/lib/python2.7/dist-packages/PIL/Image.py", line 1423, in save
    raise KeyError(ext) # unknown extension
KeyError: ''

I suspect that the above is related to the fact that some of my photos are in NEF format. I would prefer in this case to transfer JPEG.

Other frequent error (one example of many):

Traceback (most recent call last):
  File "/usr/local/lib/python2.7/dist-packages/threadpool.py", line 156, in run
    result = request.callable(*request.args, **request.kwds)
  File "./migrate.py", line 265, in move_photo
    gd_client.InsertPhoto(picasa_album, picasa_photo, filename, content_type=headers.get('content-type', 'image/jpeg'))
  File "/usr/lib/pymodules/python2.7/gdata/photos/service.py", line 427, in InsertPhoto
    raise GooglePhotosException(e.args[0])
GooglePhotosException: (500, 'Internal Server Error', 'Error: 10 STORAGE_UNAVAIL')

I verified and I do have plenty of room to upload images. Most probably Google fails, and the script should either throttle uploads, or wait for Google to come back.

And the script finished like good old C program:

Segmentation fault

Majority of pictures were not transferred up to this point.

I ran script on Ubuntu 11.04 64bit with Python 2.7, and the latest packages from PyPi.
Jack says:
Jul 16, 2011 05:26 AM
Hi Nathan, thanks for getting back to me. Turns out it was an issue with my Google Apps account (had a conflict). I've cleaned that up and the script is working much better now.

Should the image titles be set by the script? at the moment they are all being set to 'none'. Again let me know if the output would help debug.
Neil says:
Jul 16, 2011 02:40 PM
Thanks for this! Extremely useful. I'm getting this error after queuing file 398/469 in a set:

Traceback (most recent call last):
  File "migrate-flickr-to-picasa-nokey.py", line 335, in <module>
    do_migration()
  File "migrate-flickr-to-picasa-nokey.py", line 312, in do_migration
    print 'Queuing photo %i/%i, %s of album %s for moving.' % (photo_id + 1, len
(all_photos), photo.get('title'), set_title)
  File "c:\python27\lib\encodings\cp437.py", line 12, in encode
    return codecs.charmap_encode(input,errors,encoding_map)
UnicodeEncodeError: 'charmap' codec can't encode character u'\xad' in position 5
0: character maps to <undefined>

Is this a coincidence that it's on the 400th file? I could spend a day tracking this, but I figured you might know?
Nathan Van Gheem says:
Aug 10, 2011 10:35 AM
It's a unicode issue so you might want to look at the image name or album name.
Brian says:
Jul 22, 2011 02:27 AM
Do you know if your script has issues with movies >200MB? I seem to keep getting "broken pipe" messages when trying to upload large videos. I get the same error when trying to use the googlecl python utilities. Any assistance would be great. And thanks again for your script!

Brian
Nathan Van Gheem says:
Aug 10, 2011 10:35 AM
Pretty sure when I used it I had videos large than that, but I'm unsure.
Clint says:
Jul 22, 2011 07:00 PM
Awesome! One question... I used Flickr to add geotags, tags, etc. Is that data included in the migration to Picasa? THANKS!
Nathan Van Gheem says:
Aug 10, 2011 10:35 AM
I do not believe so.
Jeff Weiss says:
Aug 04, 2011 06:37 PM
For me your script hits the open files limit (which on linux defaults to 1024). I think the script is not closing the temporary files when they have finished uploading.
Dave Keeshan says:
Aug 15, 2011 06:31 AM
Hey man, interesting script. I had two issues however. Not all images have a url_o link. If the original image was too small I don't think they give it one of those links. To solve it I requested all the links and stepped down the image size until I found the biggest image, url_m usually worked.

I also found that any images not in a set wasn't downloaded and up loaded. I am investigating myself, seems need to use FLICKR.photos_getNotInSet, trying to get it put in myself but just thought I'd mention it here.

I haven't looked into it properly yet but what happens in the cases where an image exists in two different sets, does it get downloaded up uploaded twice?
Dmitry says:
Aug 21, 2011 03:03 PM
On migrating i get an error:

Traceback (most recent call last):
  File "./migrate-flickr-to-picasa-nokey.py", line 342, in <module>
    do_migration()
  File "./migrate-flickr-to-picasa-nokey.py", line 302, in do_migration
    picasa_albums = get_picasa_albums(set_title, aset, len(all_photos))
  File "./migrate-flickr-to-picasa-nokey.py", line 186, in get_picasa_albums
    picasa_album = gd_client.InsertAlbum(title=id, summary=aset.find('description').text, access='protected')
  File "build/bdist.macosx-10.7-intel/egg/gdata/photos/service.py", line 356, in InsertAlbum
  File "build/bdist.macosx-10.7-intel/egg/gdata/service.py", line 1236, in Post
  File "build/bdist.macosx-10.7-intel/egg/gdata/service.py", line 1322, in PostOrPut
  File "build/bdist.macosx-10.7-intel/egg/atom/__init__.py", line 93, in optional_warn_function
    
  File "build/bdist.macosx-10.7-intel/egg/atom/service.py", line 186, in request
  File "build/bdist.macosx-10.7-intel/egg/gdata/auth.py", line 725, in perform_request
  File "build/bdist.macosx-10.7-intel/egg/atom/http.py", line 163, in request
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/httplib.py", line 937, in endheaders
    self._send_output(message_body)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/httplib.py", line 797, in _send_output
    self.send(msg)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/httplib.py", line 759, in send
    self.connect()
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/httplib.py", line 1140, in connect
    self.timeout, self.source_address)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/socket.py", line 553, in create_connection
    for res in getaddrinfo(host, port, 0, SOCK_STREAM):
socket.gaierror: [Errno 8] nodename nor servname provided, or not known
Exception in thread Thread-1 (most likely raised during interpreter shutdown):
Traceback (most recent call last):
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/threading.py", line 530, in __bootstrap_inner
  File "build/bdist.macosx-10.7-intel/egg/threadpool.py", line 160, in run
<type 'exceptions.AttributeError'>: 'NoneType' object has no attribute 'exc_info'
Exception in thread Thread-2 (most likely raised during interpreter shutdown):
Traceback (most recent call last):
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/threading.py", line 530, in __bootstrap_inner
  File "build/bdist.macosx-10.7-intel/egg/threadpool.py", line 160, in run
<type 'exceptions.AttributeError'>: 'NoneType' object has no attribute 'exc_info'
Exception in thread Thread-5 (most likely raised during interpreter shutdown):
Traceback (most recent call last):
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/threading.py", line 530, in __bootstrap_inner
  File "build/bdist.macosx-10.7-intel/egg/threadpool.py", line 160, in run
<type 'exceptions.AttributeError'>: 'NoneType' object has no attribute 'exc_info'
Alex says:
Sep 14, 2011 10:52 PM
After a night and day of it working fine I got the following error: "GooglePhotosException: (403, 'Forbidden', 'Token expired')" until it returns "IOError: [Errno 24] Too many open files: '/var/folders/7P/7PlxwEIyFqKTLR-A9MP12U+++TI/-Tmp-/tmpMpSW1z'" and the script fails. Is there any way to continue where it left off? Thanks though! I'll definitely give it another go later on.
Vincent says:
Jan 17, 2012 04:31 PM
I get this error when running the migration:

gdata.photos.service.GooglePhotosException: (400, 'Bad Request', 'Description is too long, max length is 1000 characters.')

But how do I tell which image is causing that? The ones directly before this error do not have a description.
Mike says:
Jan 27, 2012 11:48 PM
How come certain photos won't move for me? These photos are not over 100MB, and they're definitely not videos. The Picasa album gets created, but the photos never move over, and the script just ends.


Moving "Esther Week 1" set over to a picasa album. 1/37
Gathering set "Esther Week 1" information.
Found 4 photos and videos in the Esther Week 1 flickr set.
"Esther Week 1" set already exists as an album in Picasa.
Queuing photo 1/4, of album Esther Week 1 for moving.
Queuing photo 2/4, of album Esther Week 1 for moving.
Queuing photo 3/4, of album Esther Week 1 for moving.
Queuing photo 4/4, of album Esther Week 1 for moving.
Moving "Children's Museum 2011" set over to a picasa album. 2/37
Gathering set "Children's Museum 2011" information.
Video "" of "Esther Week 1" already exists in download cache of files over 100MB. Aborting download.
 Video "" of "Esther Week 1" already exists in download cache of files over 100MB. Aborting download.
Video "" of "Esther Week 1" already exists in download cache of files over 100MB. Aborting download.
Video "" of "Esther Week 1" already exists in download cache of files over 100MB. Aborting download.
Oli says:
Feb 01, 2012 11:43 AM
Hi there,

Whenever I run the script it runs file for the first few files and then it just stops. The temp files don't seem to be downloading and I have to restart the script. Any ideas? No error message is given.
henri says:
Feb 14, 2012 10:40 PM
Hi thanks for your work I have the following error, do you have any clue?

Traceback (most recent call last):
  File "/Users/henri/Downloads/migrateflickrtopicasanokeyresize.py", line 27, in <module>
    from PIL import Image
ImportError: No module named PIL
Add comment

You can add a comment by filling out the form below. Plain text formatting. Web and email addresses are transformed into clickable links.

Navigation