Archive

Author Archive

Adding Drawing to SmartGWT Application

February 16, 2012 Leave a comment

Just a quick note to those who are trying to use Drawing in SmartGWT 3.0. If you are getting exceptions like:

com.google.gwt.core.client.JavaScriptException: (TypeError): $wnd.isc[scClassName] is undefined

Make sure you do the following:

  1. Add <inherits name=”com.smartgwt.Drawing” /> to your .gwt.xml. If you are running in a development mode with external server such as Tomcat or Glassfish, you want to make sure that this change is actually synchronized with them
  2. Clean your internet cache. If the cache is not cleared up, you will still get the exception
Advertisements
Categories: Web GIS Tags:

Choosing the optimal configurations for GeoServer

December 29, 2011 1 comment

What I wanted to do

Setup GeoServer with optimal performance. I define optimal performance as fast response with large number of concurrent users (HTTP request of WMS). The GeoServer will be serving vector line data with 80K features. The vector lines are currently stored as a feature class in ArcSDE with Oracle.

Problems

I have identified several parameters/configurations could improve the performance of GeoServer at OpenGeo and GeoServer’s documentation. In particular, here are a list of big questions that I need to answer:

1. What are the optimal JVM options

2. Which application server shall I use to serve GeoServer, Tomcat or Glassfish?

3. Which source data format shall I use to feed GeoServer? Shall I stay with ArcSDE or shall I export the feature class to a shapefile?

Here is the verision of some applications that I used:

  • Tomcat 6.0.33
  • Glassfish 3.1
  • GeoServer 2.1.1
  • JRE 1.6
  • ArcSDE 9.3.1
  • Oracle 11g

Solution

To find answers, I need to do some testing and find out what set of configurations give me the fastest HTTP response when GeoServer handles WMS request. To measure the testing result, I found an excellent tool called JMeter. It basically issues customized HTTP request to specified web service and measures the response time. It can also do the testing for multiple-user scenario.

I started with JVM options. A benchmark was established with default JVM settings, Tomcat, ArcSDE as source data format, and single user scenario. Then I changed one parameter for JVM option at a time, measured the response time and compared the result against the benchmark.

After finding the optimal JVM options, I switch the source data format from ArcSDE to shapefile and continue to run the testing on single user scenario.

Then I switched to multiple-user scenario with 10 users sending requests at 1 second interval. In this case, GeoServer’s performance decreases dramatically for either shapefile or ArcSDE as source data format.

I also did similar test with Glassfish.

Here is my findings:

Optimal JVM options: -server -Xms1024m -Xmx1024m -XX:MaxPermSize=512m -XX:+UseParNewGC. Of course, these options depend on the hardware configuration of my machine

Tomcat vs. Glassfish

  • In single user case: Glassfish is faster than Tomcat
  • In multiple user case: Tomcat is faster than Glassfish but more volatile. On average, Tomcat responses faster. But comparing with Glassfish, a lot more of Tomcat’s responses are either faster or slower, making the response time less predictable in Tomcat.
  • With user number increases, response time increases too. But Glassfish’s response time increases more than Tomcat for the same number of user increase

ArcSDE/RDBMS vs. Shapefile

  • In general, shapefile is faster than ArcSDE/RDBMS. In single user scenario, shapefile is more than 10 times faster than ArcSDE. This is reasonable because when ArcSDE is involved there is overhead of connecting to database. Even if there is database connection pooling in Tomcat, the overhead is still very costly
  • In multiple user case: response time increase for ArcSDE is less than shapefile. This is reasonable as RDBMS is designed to handle concurrent users scenario
  • In my 10-concurrent user scenario, even shapefile’s performance decreased more dramatically than ArcSDE with user increase, it still perform about 10 times better than ArcSDE. But I think with more and more concurrent users (100, 1K even 10K), ArcSDE/RDBMS’s performance should eventually out beat shapefile

So, if I want stability in performance, fast response time, AND if I predict only very small number of concurrent users, I should go with Glassfish and shapefile. In the case of large number of concurrent users, it seems that the combination of Tomcat and ArcSDE/RDBMS  should lead to better performance.

Setup GWT Development Mode with Tomcat in Eclipse

October 20, 2011 11 comments

Update

A much better solution that I found recently is to create a Google Web Project and make it into a Dynamic Web Project so that one can hot deploy the GWT stuff the same way as JSP and servlet. The example is given with Glassfish.

http://rmapper.wordpress.com/2011/02/02/gwt-development-mode-with-hot-deployed-glassfish-in-eclipse-2/

forget about the following:
Old crap

After developing GWT application with NetBeans for several weeks, I found that I need to give up NetBeans and switch to Eclipse.

On one hand, to develop GWT application with NetBeans is easy to start. A GWT4NB plugin and necessary GWT SDK will allow you to build GWT apps. Debugging with Tomcat is extremely straightforward. Since I already have Tomcat setup for NetBeans, there was nothing in particular that I did after installing GWT4NB and GWT SDK to be able to debug GWT application with Tomcat. Moreover, my GeoServer for development has been running on Tomcat. Although I could take advantage of the embedded GWT development mode with Jetty, I really want my GWT application to run on localhost:8080 with Tomcat so that I can issue HTTP request from GWT app to GeoServer.

On the other hand, GWT Designer doesn’t work with NetBeans. This has not become an issue until I started to build complex GWT widgets using SmartGWT. Without GWT Designer, it was overwhelmingly painful to tweak the GWT application (which remind me my good old days with Visual Studio).

Problem: Setup Eclipse to include GWT and configure GWT in Eclipse to work with Tomcat at development mode

Solution:

1. I followed Set up Eclipse instruction to install Eclipse and include GWT

2. Go to Compile & Debug to find instruction to debug with my own server. The instruction’s step-by-step description of how to use -noserver confused me. So here are the steps that I have summarized to hopefully make a starter like myself understand better. Basically 1) you need to setup a place holder in Tomcat so that Eclipse can deploy your modified code to Tomcat in the GWT development mode, and 2) you need to tell Eclipse where to deploy in the GWT development mode.

  1. Compile your GWT project in Eclipse with Google Plugin for Eclipse: Right click on your project -> Google -> GWT Compile
  2. Create place holder on Tomcat. After compile, find your project folder in Eclipse workspace. Locate YourProject/war directory. Copy all stuff under the war directory to the webapps folder under your Tomcat installation. You need to create a new folder under webapps to hold the stuff copied from war directory. You are essentially deploying your compiled GWT project to Tomcat
  3. Configure GWT development mode. In Eclipse, right click on your project -> Debug as -> Debug Configurations -> Web Application -> your project -> Go to Server tab -> uncheck “Run built-in server” -> Go to GWT tab -> in URL, put the URL from which you access the project deployed on Tomcat. For example, http://localhost:8080/myproject/index.html
  4. Click Debug. Now you should see that the development mode generates URL based on http://localhost:8080.
  5. Now launch the development mode URL and try to modify some contents in your project source code. For example, change the html title name and save. After that, refresh the URL on browser. And you should see your changes applied to your deployed project on Tomcat in development mode.

Note that at this point the development mode will only transfer the code that you have modified under src/. Other so called static contents such as your .html and .css would not be transferred to Tomcat. What you might have noticed is that if you edit .css file under war directory, you edits won’t go to Tomcat. My work around is to create an ant script under my project in Eclipse and ask the script to copy .html and .css to the webapps/myproject on Tomcat. Now every time I make change to .html and .css, I will run the ant script before refreshing the webpage.

Categories: Web GIS Tags: , , ,

Getting Rid of Orphan State Locks in SDE with Oracle

August 31, 2011 Leave a comment

Problem

User of SDE database with Oracle was doing editing on a featureclass on ArcMap when ArcMap suddenly crashed. After the crash, a state lock stayed in STATES_LOCKS table that prevented SDE database compress to state zero. When SDE database can’t be compressed to state zero, the edits that user made on states other than state zero would NOT be moved to business table after compress. Once this happens, most likely you will find that the user who made the edits can see the change on his feature class, SDE admin will get successful result running sdeversion -o compress, but the changes user made at his end just won’t show up on web map that consumes the feature class.

Solution

I ran into similar problem recently. There are several things you can do to fix this issue:

1. Check to make sure ALL your database users have been disconnected from ArcCatalog or ArcMap. You can first look at SDE_ID in STATE_LOCKS table. Then run SDEMON -o status -I users. This command will give you the users that is connecting to your SDE database. You should find an ID in the first column of the command result. Look for ID the same as SDE_ID in STATE_LOCKS table. This way you can tell which user (most likely the PC of your user) is giving you the trouble. You can run SDEMON -o kill to kill an inactive connection.

2. If you still see a state lock, that means most likely this is an orphan lock. DON’T try to use SQL to remove the state lock because using SQL to manipulate these tables have known to crash SDE database. Instead, run SDEMON -o shutdown to stop your SDE service and then SDEMON -o start to restart the service. After doing so, you might still see the state lock but once you issue sdeversion -o compress to compress the state tree, the orphan state lock will go away.

If you are as unfortunate as me, you might run into an issue where after stopping your SDE service, you can’t start it – SDEMON -o start just won’t response. In this case, ask your UNIX admin to find additional active process on the server that belongs to SDE user and kill it! In my case, a “oracle” process is still running and it is manually killed. After killing the process, SDEMON -o start works again.

Categories: Spatial Database Tags: ,

Unable to delete representations – Problem solved

What I wanted to do

Test Representation in ArcGIS

What I have

  • Oracle 11g
  • ArcSDE 9.3
  • ArcGIS for Desktop (according to recent ESRI product naming announcement)

Issue

Here is what happened:

I was learning how to use representation. So I created a representation on a feature class in my SDE database on Oracle. Once the representation is created, in ArcCatalog I looked into the attribute table of the feature class and saw two columns: ruleID and override. I right clicked on the columns and delete them. And then I went to the properties of the feature class and delete the representation from Representation tab.

Now annoying things happened: I restart ArcCatalog and open the properties of the feature class, navigate to Representation, nothing shows up in the list, which is normal because I created and then deleted the representation. But if I click new and create a new representation, the one that had been created and deleted would show up together with the new representation in the list. If I select the ghost representation and try to delete it, I get the Unable to delete representation notice. What’s worse is that this ghost now shows up in all OTHER feature classes representations tab and I can’t get rid of it! Even if I remove this feature class the ghost representation is still there.

Solution

I haven’t seen similar issues posted on the Internet but the solutions to  many representation issues involve do something about an Oracle system table named SDE.GDB_ExtentsionDatasets. So I looked into this table in Oracle and found the dead body of that representation ghost. After removing the record, the ghost went away!

Wrap data around 180 longitude with ArcGIS Server and JavaScript API

What I wanted to do

I have data extends across 180 longitude. When I build a web map out of the data, I want it to be wrapped around 180 so that I can pan the map across 180. To achieve that, according to samples in ArcGIS JavaScript API 2.3, I need dynamic map service published by ArcGIS Server 10 and add “wrapAround180:true” to the options of esri.Map.

What I have

  • Shapefile
  • ArcGIS 10
  • ArcGIS Server 10

Solution

First of all, what is dynamic map service? My understanding of dynamic map service after reading Publish Map Service in ArcGIS 10 Desktop Help is that as long as the map service is not cached, it is dynamic map service. So I followed the steps listed in the help to publish my data as a map service with WMS, with caching set to “Draw this map service dynamically” in the service setting in ArcGIS Server. Then in the web map that I built with ArcGIS JavaScript API, I set the map to wrap around 180 (var map = new esri.Map(“MyMapID”, {wrapAround180:true})) and created a layer consuming the WMS (var wmsLayer = new esri.layers.WMSLayer(“the WMS layer”)).

It didn’t work. The map was still cut off at 180.

After some internet research, I found out that to make the wrap around 180 work, I need to create a ArcGISDynamicMapServiceLayer and the layer has to consume a REST service. So now the question became how to create a REST service.  According to ArcGIS Server REST API, to find out the REST services on my ArcGIS Server, I used http://<host&gt;:8399/arcgis/rest. My data was not there, even through I have published my data as map service. After talking to someone who have experience with REST service, I found out that the problem is related to REST service caching. On my ArcGIS Server, the caching has to be cleared manually, which is why my data didn’t show up in the REST service directory. To manage the REST service, I used http://<host>:8399/arcgis/rest/admin. Once the cache is cleared manually, my data showed up.

Now, instead of a WMS layer, I create a dynamic map service layer in my web map (var restLayer = new esri.layers.ArcGISDynamicMapServiceLayer(“http://<host&gt;:8399/arcgis/rest/services/MyData”)) and guess what, my data goes across 180 now!

How Google Fusion Table made my life MUCH easier


Google Fusion Table is still in Beta. But it’s powerful! There are so many things you can do with Google Fusion Table and as far as I am concerned it has THE potential in many ways to redefine how GIS professionals mange and publish GIS data.  I am going to give you one little example to demonstrate that and show you how it made my life much easier.

What I wanted to do

Build a web map that consumes my GIS data. I wanted user be able to click on the feature on web map to get attributes of the selected feature

What I have

  • My GIS data in ESRI Shapefile
  • WMS of my data served out by GeoServer
  • Email from my department telling me that ArcGIS Server is too expensive to purchase

Issue

I built a web map with OpenLayers, and Google Map as background. It consumes WMS served out by GeoServer. When user clicks on the feature on web map, the attributes of clicked feature are displayed on the webpage below the map. This seems to be fine. But:

  1. A proxy to handle cross-site scripting problem was introduced into this web map so that when user clicks on the feature, the attributes actually show up. This is a known security risk and even though the proxy mechanism works in a test environment, my organization won’t allow this proxy thing to happen when the web page goes public
  2. I spent lots of effort to program OpenLayers, Google map, WMS and the CGI for proxy on my web page so that they work well together but in the end all these efforts are wasted because of the security risk

Solution

I tried Google Fusion Table and its a perfect way to work with in my situation.  Google Fusion Table allows you to do the following cool things:

  1. Upload spatial data in KML format
  2. Plot uploaded spatial data on Google Map
  3. When clicking on the spatial data on Google Map, you get popup of the attributes

So I use ArcGIS to convert my shapefile into KML and upload the KML into a fusion table. Once my data is in a fusion table, I clicked on Visualize -> Map to get a Google Map view.  On top of the Google Map, I can “Get embeddable link” of my data in a Google Map view. Then I use this link on my web page for the web map. See the bottom of this post for how my data looks like. Just in case you haven’t realize the huge benefit of using Google Fusion Table in my situation:

  • I don’t need a sever
  • I don’t need to program OpenLayers and WMS etc.
  • I don’t need to worry about cross site scripting security risk. Speaking of security risk, my data is not sensitive so I have no concern of data security

I highly recommend anyone who works in professional GIS arena try Google Fusion Table. It is definitely redefining how we publishing and sharing GIS data.