Web App & Extension: Nov 9. 2011: Even more bad news
Wednesday, Nov 9. -- It's over.
This project is dead.
You can keep it installed if you want to, but it won't be the same.
1) The server which gathers the data from USGS only gets updated once per day, at midnight San Francisco time.
2) The server which gives the data to the clients only works during the first 14 hours of the day, unless all of you would deactivate the "automatic update" function.
Web App & Extension: Nov 8. 2011: More bad news
Tuesday, Nov 8. -- We're still down!
Things are looking bad.
The Web App and the Extension rely on two server hosted on Google App Engine. Initially I thought only one server was having problems, but it turns out that both of them are.
One server is there to serve the data to the App and the Extension. It looks at the last update ID the client sends to it, and if it is obsolete, it serves him the latest data, else it serves an empty document, in order to save bandwidth. Also, if the client requests more data than the server has, it redirects him to another server which has that data.
This simple step is apparently so CPU intensive, that the server reaches it's free quota within a couple of hours. Google is kind enough to give us USD 1 per day of CPU time for free, but this is not enough. When the server reaches the quota limit, it stops serving any data. I'm using about USD 2.3 per day, which makes it about USD 70 per month, add the USD 10 of fixed cost and we're at about USD 80 per month for that one server.
The second server is the one which fetches the data from USGS, analyzes it, stores it, and then pushes the updated data to the other server. This server now get's blocked almost immediately due to a short term quota limit. This basically means that it has become useless.
If I'd enable billing for this server, I'd had to pay USD 0.44 per day, which is about 13 USD a month, plus the fixed cost of USD 10, which turns out to be a total of USD 23. Plus the other server makes it all cost about USD 100 per month.
Web App & Extension: Nov 7. 2011: Bad news
Monday, Nov 7. -- We're down!
I'm working on it, but it will take a couple of days.
Until this issue has been resolved, the Apps will remain unpublished (not listed in the Webstore).
Here's the issue: Google App Engine has just come out of beta, with a new pricing model in place. This pricing model is too expensive for me to continue offering this service for free. I'd probably need around USD 30 per month to keep it running as it is. I'm in talks with Google to see what we can do about it without relying on donations.
Web App: Oct 26. 2011: A note about the Web App
First of all, thank you for your kind reviews, and a thank you to Google, who gave me a Chromebook as a sign of appreciation for the work I've been doing on this extension. While the extension didn't fit the requirements of the Challenge, it still got the most points from the jury
(see here and here).
Development on this app hasn't stopped, but the next changes will be so big, that there is a heavy amount of coding going on here. There are two major changes:
1) The map. It's going to be resizable, and the inline labels will be clearer and more informative.
2) The data source. Currently the USGS information is being gathered by a backend server on Google App Engine, from where each client is fetching the information. This leads to a delay which can be as huge as 19 minutes (15 minutes server side, 4 minutes client side). This is not acceptable. So I'm changing the clients to fetch the data directly from USGS. This will increase the load of USGS, and let's hope that they are ok with this, but it will drastically decrease the time between USGS publishing an earthquake and you getting notified about it. Down to probably one or two minutes, depending on how much strain we want to put on USGS' servers (I may need to get in contact with them).
There are also some other changes, like an integrated BBC radio player, as well as an Al Jazeera live video stream, just in case.
While I still don't know when I'm going to release this, here are some images of the new interface:
In the image above, you'll see that it is still pretty similar in layout to the current version (timeline is still not implemented; but at least I already know how to do it).
The maximized map view really let's you see the surroundings of the epicenter. In this image, and the ones below, you can notice the yellow labels as well as the green one. The green one is super useful if you start to zoom in a lot an explore the surroundings. It always shows you how far you are, and in what direction the epicenter is.
Web App: Version 1.9.4 - May 26. 2011: A note about the Web App
Lately I focused on the development of a web app, which is the evolution of this extension. This does not mean that the extension will become obsolete. It just means that I had no time to improve the extension. The web app is very similar to the extension, and the code will be merged when I find some time to do that.
The new features which currently are in the web app but not in the extension would have been implemented directly in the extension, had Google not launched their developer challenge (which BTW has been delayed).
If you want to use the new features, use the web app in the meantime. It's just too bad that web apps don't support badges, the little icon in the navigation bar. But the news and twitter pane are really worth it, so you might want to check it out. If you do, please write a nice review for the new Web App in the Web Store if you feel so; it could help me win a little prize.
BTW, you need to be logged into the Web Store in order to install the app. But since it's a free app, you're not handing over any data to me. It's just a Google thing.
For those who didn't notice that this is possible to pan and zoom the timeline, as well as to select earthquakes in it:
Extension: Version 1.2.7 - April 15. 2011
1) Fixed the zooming issue which occurred when no cities where configured.
2) Fixed the missing circle issue which occurred when a non-stable version of Chrome is used. Looks like Google changed the rendering code for Google Maps, and now the circles are no longer nicely round as you can see in the screenshot :(
3) Added an info overlay to the map.
4) I also added the option to skip the popup entirely so that when you click on the badge the tab get's focused if it was already open, and if it wasn't open it get's created. I like it a lot, I've quit using the popup entirely.
5) I added a link to the options page which allows you to create a bookmark in order to use it to open the tab.
Extension: Version 1.2.6 - April 11. 2011
1) I made changes to the permissions. This is what they look like, and why I want them:
Tabs (browsing history)
I need the tabs permission in order to open new tabs. The extension has the ability to be opened inside a tab (like a normal webpage), which makes it work like a web app. This allows us to open a chrome window with the earthquake notifier opened in it while continuing using the computer normally. Else the popup would close because the window loses focus and we'd have to click on the icon in order to be able to look at the info instead of just catching a glimpse at the open window. The warning is really annoying, since the API docs state it is possible to create new tabs without using this permission, but it does actually not work without it. The ability to view the "browsing activities" is a result of the tabs permission. There's nothing I can do about it.
Geolocation (physical location)
I want (not need) this permission in order to implement two features in the future: 1) Collecting our Earthquake Notifier configuration (our location and which cities and countries we observe, and the date and time when we add or remove them, as well as - if possible - the frequency which we will be looking at earthquake events) in order to create an animation of the behaviour of a swarm of anonymous users over time in the case of a really strong earthquake. 2) In order to automatically offer additional services which are related to our proximity to an earthquake, like the USGS "Did you feel it" website. Should you disagree, then feel free to uninstall the extension.
Data on eqn.danielfaust.org, ext-earthquake.appspot.com and ext-earthquake-server.appspot.com
Currently the appspot.com domains are serving the earthquake data, but they are being moved over to the eqn.danielfaust.org domain. What used to be an "can access all data on all websites" warning, has now been narrowed down to these 3 domains. I didn't restrict them before not because I needed access to all the data on all websites, but just because I didn't care about it. No harm has been done.
2) I upgraded Google Maps to version 3 and put a good deal of effort into making the Google Maps pane more informative. It now contains circles demarcating the radius of 50km, 100km, 200km and 400km (50mi, 100mi, 200mi and 400mi in case you're using miles) around the epicenter and scales and repositions itself better. The red area lies within a 100km radius, the yellow one within 200km (or miles, respectively)
3) I also added the option to skip the popup entirely so that when you click on the badge the tab get's focused if it was already open, and if it wasn't open it get's created. I like it a lot, I've quit using the popup entirely.
4) I added a link to the options page which allows you to create a bookmark in order to use it to open the tab.
Extension: Version 1.2.5 - March 26. 2011
Friday, March 25: Just noticed that I completely broke the city configuration in the options page in version 1.2.4. I will fix this tomorrow. Please don't try to add/remove/modify cities until the next release or you might lose your configuration.
Saturday, March 26: Version 1.2.5 now fixed that bug. If you were getting a white popup and the cities list was missing in the options page, it should repair itself. If you deleted a city, the wrong one might have gotten deleted by the software. In that case you will need to add it again. I'm sorry for that.
Extension: Version 1.2.4 - March 25. 2011
1) When viewing the content inside a tab, no more size constraint exist.
2) The popup dimensions can be configured manually in the options page.
3) Timezone updates now work correctly. Please go to the options page and click "Update Timezones" once to make sure that they are up to date.
4) You can modify the timezone values manually, in case that the geonames server returns outdated data or is unreachable.
5) You can modify the latitude and longitude data manually.
6) The timeline visualization has been redesigned a bit, and the selected node now shows the closest city to the earthquake as well as its distance to it.
7) Visual feedback is given when changes in the options have been saved.
8) Upon your request the browser can check your location to help you adding your location to the list of cities you are observing.
Extension: Version 1.2.4 - March 17. 2011
The announcement that a version 1.2.4 is avaliable is an error. It was an mistake which resulted only in an announcement, but no new version 1.2.4 will be uploaded now, so please ignore this message.
It occurred because I'm optimizing the backend on Google App Engine, which is having a very high CPU load because of the horrible events occurring in Japan. The notifications are currently hard coded into
the backend, and since I already prepared the announcement of version 1.2.4 in the backend code, it slipped out into public when I pushed out a new, optimized version of it.
Extension: Version 1.2.3 - March 16. 2011
Decreased the popup height so that it fits into a 800x600 screen (I will put this into an option variable in a future release, so bear with me if you think it's too small)
Extension: Version 1.2.2 - March 16. 2011
1) Moving the timeline with the mouse works again (it broke on some update of google chrome).
2) Opening the popup into a tab now expands the content to the entire page and shows an icon (good if the tab get's pinned).
3) The slide in notification now has a well formed layout.
5) Opening the popup is now a little bit faster.
6) The sound file is now included in the installer (previously it was streamed from a webpage due to a bug in chrome).
Extension: Version 1.2.1 - April 19. 2010
Fixed a bug in the options page where for some users the map was not working, and thus preventing them from configuring the cities.
Extension: Version 1.2.0 - April 14. 2010
Fixed a bug which was affecting some but not all users. I will focus on the backend from now on, since that one has still a bad issue.
Extension: Version 1.1.17 - April 12. 2010
As you may have noticed, I pushed out a new verison yesterday.
I added a feature to delete unused earthquake data. There are two deletion options:
1) delete earthquakes that are weaker than your "ignore earthquakes weaker than"-settings and
2) delete earthquakes older than the oldest visible day that you've set in your options.
That way I managed to reduce the earthquake count on my Netbook from about 6500 earthquakes to about 700. But displaying them is still not faster :(
I also added 2 debugging things, one is a Debug ID, and a new page which shows you a summary the content of the earthquake data stored locally in your browser.
I removed a bug which prevented the timeline to be scroll- and zoomable.
As you may have noticed with yesterdays earthquake in Spain, if you have your setting of "show earthquakes with a magnitude above" set to 6.0, it appears twice. I've tracked down the bug in the backend, and it needs some rewriting since there is a serious design flaw in the algorithm. That data will get corrected as soon as the algorithm has been fixed.
Also, if you check for new earthquakes very often, you'll notice that the newest earthquakes appear, dissappear and reappear occasionally. This is a problem because the backend is getting different versions of the eartquake data from fetch to fetch, and there's nothing I can do about it. I'll try to figure out something, but it has a low priority.
Extension: Version 1.1.16 - April 10. 2010
I removed a bug in the options section where a popup would appear when timezone information was succesfully fetched from geonames.org (it was a remainder of an debugging attempt, because there were issues with geonames.org and I had to find out where they came from. It turned out that geonames.org had a hard disk crash and thus were offline for two to three days.)
I also changed the hovering mode a bit. Now the hovered earthquakes don't appear bold, but oblique. This let's new earthquakes to be bold and therefore more easily distinguishable. Looks nice, right? I wish I had a screenshot of how it looks on a Mac, because I don't know.
Another new feature are popup notifications. If you enable them in the options, whenever new earthquakes come in, you get a popup dialog showing you a short summary.
Extension: Version 1.1.15 - April 8. 2010
Due to a comment of a user, the hovering modes have been modified. If you have no cities configured in the options, then, while hovering an earthquake, the earthquake is always centered in the map, instead of the lat-long 0,0. If cities exist, then the region name column centers the map at the city closest to the earthquake, and tries to zoom in as much as possible. If the mouse hovers a city column, the city is always centered in the map.
The options page now fails a bit more gracefully if geonames.org is not accessible (as it is currently :( ), and there are two helping links added for manual configuration: the country code list on Wikipedia and a link to a timeanddate.com for timezone information searches.
Until now, each time the extension looked for new quakes, it downloaded and checked a list of the eartquakes which occurred during the last 25 hours. Now the server only returns that list if new earthquakes are present, that makes the check much faster, and saves a lot of bandwidth on the server.
The server is currently serving approximately 350 users, and half of them are fetching data at any given time. That makes up for about 25% of the free bandwidth quota provided by Google App Engine, as well as 12% of the free CPU usage quota. Let's see how that change affects the resource usage on App Engine.
Extension: Version 1.1.14 - April 5. 2010 (1.1.13 has never been released, but erroneously announced)
The backend had an issue where modifications older than 24 hours to the data from part of the USGS would not be stored on the server. Now the data is always up to date. All data reaching back to March 29th has been corrected on the server, data before that date will still contain a couple of errors.
When Google Chrome is started, the extension checks for all changes made during the last week. All subsequent requests for data will only download the last 24 hours, so it may happen that modfications made to data older than 24 hours will only be corrected when Google Chrome starts again. This will get optimized in the future.
There was an issue with the timezone information which has been fixed now, so you should go to options and click on "Update all timezones" so that the correct timezones are set.
The permissions have been changed from http://ext-earthquake.appspot.com, http://ext-earthquake-server.appspot.com, http://ext-earthquake-server.soundlinker.com to any domain. This has been done so that future modifications can be rolled out more easily as there will be a social feature added to the extension.
Twitter has been removed as a news link, as the data provided by it is relatively irrelevant.
Due to the high amount of earthquakes registered today in Northern Mexico and Southern California, the backend will only update it's data every 6 minutes (by checking USGS for changes), this will stay like this for a couple of weeks, but will get optimized in the future.
Extension: Version 1.1.12 - March 21. 2010
You now can
1) update the timezones of the cities you have already added by clicking on a button.
2) reorder the cities so that they appear where you want them to be.
3) rename the cities.
5.4 Note: Solved: If earthquake data is modified by USGS after a period of 24 hours, those changes won't show up.
This issue will be fixed, but it's not an easy one so it will take some time.
For example, this earthquake Magnitude 6.6 - NEW IRELAND REGION, PAPUA NEW GUINEA
was submitted as a magnitude 6.2 earthquake, but got changed to a 6.6 after having been reviewed.
Changes that are being made within a 24 hours period are corrected automatically.
This bug wouldn't have been noticed if it were not for the cool Social Media Monitoring extension!
19.3 Note: Added sound notifications. Enabled by default, can be disabled in the options page.
18.3 Note 4: Added a blue "page" link to the popup which allows to open the popup in a separate tab, so that you can keep it open while surfing in other tabs.
18.3 Note 3: Added links to Google News and Twitter Search for each earthquake.
Improved popup show up speed drastically in 1.1.8.
18.3 Note 2: New App Engine application is up and running! The old one refers the clients to the new one, which has "static" up-to-date data sets of the earthquake data. As soon as the old one has gathered and processed new data from the USGS website, it pings the new one so that it can fetch the latest data from the old app and then serve it to the clients. I also decreased the update interval from 15 to 5 min. I'm hoping that everybody get's updated to the latest version within the next 6 hours, else the quota will be depleted again for today.
18.3 Note: There are some serious issues with the backend, which wasn't able to serve for the last 3-4 hours, because the free CPU usage limit on Google App Engine was exceeded. I've temporarily modified the update interval to 15 minutes, until I implement a good caching strategy on the backend. You can check the status of the backend here http://bit.ly/cQN710
You can always force a manual update by clicking on the update link in the popup, as long as the backend is working.
17.3 Note 2: The issue below (which got deleted...) has been fixed, was apparently broken since yesterday.
( geonames.org was DDoS'd, see http://bit.ly/9fIbak )