Deploying a static website efficiently on Google Cloud Storage

I have several static websites hosted on Google Cloud Storage for which the only thing missing is the ability to have a custom domain using SSL.

In my deploying process from the local file system I was using the gsutil cp command that enables gzipping the files by extension (html,css,js). However each time every file was copied which was a pain when I was using several unmodified images.

My first experience with gsutil rsync was bad:

  • I could not gzip the files I needed
  • Just touching a file was causing it to be re-deployed
However I was able to find an optimized way for deployment
  • Split my local folder into 2 folders with the same hierarchy, one containing the content to be gzip (html,css,js...), the other the other files
  • Gzip each file in my gzip folder (in place)
  • Call gsutil rsync in for each folder to the same gs destination
Of course, it is only a one way synchronization and deleted local files are not deleted remotely
For the gzip folder the command is

gsutil -m -h Content-Encoding:gzip rsync -c -r src/gzip gs://dst

forcing the content encoding to be gzippped

For the other folder the command is

gsutil -m rsync -c -r src/none gs://dst

the -m option is used for parallel optimization. The -c option is needed to force using checksum validation (http://stackoverflow.com/questions/39016621/why-is-gsutil-rsync-re-downloading-all-our-files/39996032#39996032). the -r option is used for recursivity.

To do all this, I wrote a script (in dart of course !) that you can try

# Install the tool
pub global activate -s git git://github.com/tekartik/tekartik_deploy.dart

Create a simple web project

# Build
pub build
# Deploy Install my build folder to my bucket
gswebdeploy build/web gs://my_bucket/my_folder

You should see the following

Building synchronization state...
Starting synchronization
Copying file://build/gs/web/gzip/index.html [Content-Type=text/html]...
Copying file://build/gs/web/gzip/main.dart.js [Content-Type=application/javascript]...
Copying file://build/gs/web/gzip/packages/browser/interop.js [Content-Type=application/javascript]...
Copying file://build/gs/web/gzip/packages/browser/dart.js [Content-Type=application/javascript]...
Copying file://build/gs/web/gzip/styles.css [Content-Type=text/css]...          
- [5/5 files][ 12.5 KiB/ 12.5 KiB] 100% Done                                    
Operation completed over 5 objects/12.5 KiB.  

If you run again, you'll see that only modified files are uploaded

Popular posts from this blog

Fixing "DerInputStream.getLength(): lengthTag=109, too big" in Android Studio

This happened to me in Android Studio after AndroidX migration and using the new testing framework. Even deleting the existing ~/.android/debug.keystore was failing for me The solution was regenerate it manually (accept all questions as empty and say yes at the last one) $ keytool -genkey -v -keystore debug.keystore -storepass android -alias androiddebugkey -keypass android -keyalg RSA -keysize 2048 -validity 10000 And copy it $ rm ~/.android/debug.keystore $ cp debug.keystore ~/.android/debug.keystore
Read more

Simple Samba setup on Lubuntu

I have a simple lubuntu server where I setup a ssh server. While it is sufficient to access a driver/folder from another linux machine using sftp, it is a pain from Windows & Mac, so unfortunately, samba is still needed here. For simplicity I needed only one samba user that I could map to a single user on my lubuntu machine Following the various tutorial I ended up doing this with simple commands Install samba $ sudo apt-get install samba Add myuser to the samba $ sudo smbpasswd -a myuser Export the drive I needed (no guest user) $ sudo vi /etc/samba/smb.conf to add the following export (no real option needed here) [files] path = /media/music/files writeable = yes save and Restart samba $ sudo service smbd restart
Read more

Solving: insufficient permissions for device: user in plugdev group; are your udev rules wrong

This happens on Ubuntu almost each time I add a new device: Check that your are in plugdev group $ id uid=1000(alex) gid=1000(alex) groups=1000(alex),4(adm),24(cdrom),27(sudo),30(dip),46(plugdev),116(lpadmin),126(sambashare),127(kvm),129(libvirt),134(chrome-remote-desktop) If not (i.e. plugdev not displayed), add it: $ sudo usermod -aG plugdev alex If you get 'insufficient permissions for device: user in plugdev group; are your udev rules wrong' when trying to debug, you'll need to add your device: First i used lsusb to find my vendor id (here I guess google) $ lsusb Bus 002 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub ... Bus 001 Device 025: ID 18d1:4ee7 Google Inc. .. Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub .. Bus 001 Device 020: ID 05e0:2106 Symbol Technologies USB2.1 Hub As suggested in https://developer.android.com/studio/run/device.html the solution was to sudo gedit /etc/udev/rules.d/51-android.rules Add th...
Read more