Tuesday, April 26, 2016

street intersection


1) unsplite street to remove segment, (no advance license, can't use this tool)
    alternative is use dissolve tool (data management  > generalization > dissovle)





check unsplit lines,
uncheck create multipart features,

dissolve field, by street name.




2) analysis tool > over lay > intersect

 Run Intersect tool on a polyline feature class itself; specify POINT as Output Type. This will give you duplicated points at intersections only (not dangles).



 3) Run Delete Identical tool on the points, specify SHAPE field. Now you should have ONE point at each intersection.(no license) 
alternative is use Qgis,  menu > MMQgis > modify > delete duplicate geometries

      

4) run spatial join (analysis tool > over lay > spatial join)


add output field,  merge rule: join,  delimiter:  "  &  "
match option: intersect. 
Detail see below blog




5) now you have a field named 'intersection' as  "streetA  & streetB & streetC"
you want to use vbscript to split by & into array.

for street 1: 
pre-logic script code:
    a = Split([inter], "&")

str_1 = 
    a(0)



for street 2:
pre-logic script code:
--------------------------------
a = Split([inter], "&")

b = UBound(a)

If b>0 Then 

st2 = a(1)

Else

st2 = ""
End If
------------------------------
str_2 = st2
--------------------------------------


for street 3:

pre-logic script code:
----------------------------------------
a = Split([inter], "&")

b = UBound(a)

If b>1 Then 

st3 = a(2)

Else

st3 = ""
End If
-------------------------------------------------
str_3 = st3
---------------------------------------------------------





for street 4:
=======================
a = Split([inter], "&")

b = UBound(a)

If b>2 Then 

st4 = a(3)

Else

st4 = ""
End If
=============


for street 5:
=================

a = Split([inter], "&")

b = UBound(a)

If b>3 Then 

st5 = a(4)

Else

st5 = ""
End If
====================


6) create 2 street intersection file, one with field a street and b street, another one with field reverse as b street and a street. 

file: 
Street_intersect_a&b ,  intersect = [str_1] + " & "  + [str_2]


file: 
Street_intersect_b&a ,  intersect = [str_2] + " & "  + [str_1]

use append point feature to combine above 2 files.

data management tool --- general ----- append  





+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

https://blogs.esri.com/esri/arcgis/2013/02/11/streetintersectionlist/
===============================================================
If you work in local government, your GIS shop is most likely responsible for maintaining a street centerline database.  And if you’re the keeper of the street centerline database, you’ve probably had requests to create a street intersection list—a point feature class with attributes that list the names of the cross-streets. Typical uses for a street intersection list is inventory, inspection, and management of signs and other assets, as well as for input to E911 systems.  They are also useful for quality assurance of the street centerline database.
Creating a street intersection list is an interesting exercise in overlaying mixed feature types, in this case points and lines, as well as dealing with one-to-many relationships, so I thought it worthwhile to work through the steps as part of this series of blogs about overlay.

Download

You can download the data and models used in this post here.

Strategy

The basic strategy is to first create a feature class of points where streets intersect. Then, using some sort of spatial search, construct a list of all the street names connected to each point. The last step can be done using the Spatial Join tool (in previous posts, I showed how to use Spatial Join and its Field Map parameter to create such an attribute, and I’ll use the same technique here).
For my first attempt at creating a point feature class, I used the Feature Vertices To Point tool, which seems to be the favorite starting point according to most message boards. Feature Vertices To Point creates a point for the first and last vertex of every street feature. The problem with this method is that it resulted in many more points than I needed. For example:
  • One point is created for each line connected at an intersection. That means for a four-way intersections, four identical points are created where only one is needed. These extra points just get in the way and can slow down processing.
  • Points are created at the ends of dead-end streets, and I’m not interested in dead-ends since they aren’t intersections—there’s only one connected street.
  • Finally, points are created where two street features with the same name connect. This splitting of street features is typically due to an attribute change, such as a disconnect in address ranges as shown here, or a change in road class or some other non-name attribute. The term I use for these intersections is ‘pseudo-junction’. Since the street name doesn’t change at these pseudo-junctions, I’m not interested in them either. (The split is hardly ever due to a change in street name—except here in Redlands where we seem to have lots of streets that change names for no apparent reason.)
Although the Feature Vertices To Point method gave me a starting set of points, I wanted a method that gave me a smaller starting set and less artifacts (dead-ends and pseudo-junctions).

Unsplitting lines

The first step is to run the Unsplit Line tool using the street name field for the Dissolve Field parameter. Unsplit Line merges connected line features with the same street name, as shown below, where the red points represent the endpoints of the line features. The output has far fewer line segments then the input, and pseudo-junctions are removed.

Create intersections using the Intersect tool

The output of Unsplit Line is input to the Intersect tool, as shown in the model snippet below. Intersect creates points where line segments touch.
  • When Intersect is run with one input feature class, the feature class is intersected with itself. You don’t need to (or want to) input the same feature class twice.
  • The Output type is POINT, meaning that the output features are points created where two streets intersect. The default value forOutput type is INPUT, and if you specify INPUT, the output will be an empty line feature class.
  • In this instance, the JoinAttributes is set to ONLY_FID, meaning that no attributes from the input features will be carried to the output feature class.

Removing duplicates

The output of Intersect is a feature class with duplicate points at every intersection. Note that there are no points created for dead-ends, nor are there any pseudo-junctions.  To remove the duplicates, keeping only one of the duplicate points, use the Delete Identical tool as shown in the model snippet below.  For the Fields(s) parameter in Delete Identical, use the Shape field.  This deletes all but one of the points that share the same x,y location. (The Delete Identical tool requires an Advanced license.  If you don’t have Advanced, use the Collect Events tool instead.)

Finding the intersecting streets with Spatial Join

The next step is to use the Spatial Join tool to create an attribute on the intersection points that lists all the streets that connect to the intersection.
This usage of Spatial Join depends upon defining a new output field (“Names”) in the Field Map of Join Features parameter. The first thing to do is to remove all the input fields from the Field Map control by highlighting the field and clicking the remove button. Then follow these steps, all illustrated below.
  1. Right-click within the Field Map control and choose Add Output Field. The Output Field Properties dialog box opens.
  2. In the Output Field Properties dialog box, name the field “Names”.   Do not use the same name as your input street name field (the one used in the Unsplit Lines tool). Choose Text for the Type of field, make its Length big enough to hold the concatenated field names (I chose 80).  Choose Join for the Merge Rule and provide a delimiter ‘ & ‘  (space/ampersand/space). Click OK.
  3. The field map will show this new output field. You now have to tell it what fields you want to join by right-clicking Names and choosing Add Input Field. This opens the Add Input dialog box.
  4. In the Add Input dialog box, select the street name field (FULLNAME)  and click OK. The field map will now show the new field along with the input field that will create the values for the new field.
  5. For the Join Operation parameter, choose JOIN_ONE_TO_ONE.  For the Match Option, choose INTERSECT or WITHIN_A_DISTANCE.  For either option, you can specify a Search Radius.  One meter is plenty.
The output of Spatial Join is shown below.  For each point, the intersecting streets are found in the Names attribute, each street separated by the delimiter (an ampersand ‘ & ‘ in this case).  The Spatial Join tool creates the Join_Count attribute for you.  This is the count of intersecting streets.

Exploring the results

The output of Intersect is a multipoint feature class, meaning that one feature can contain more than one point.  The majority of the intersections have just one point, but if a feature contains more than one point, it means that the streets intersect more than once, as shown in the figure to the left.
To find features with more than one point, use the Add Field tool to add an integer field, and use the Calculate Field tool with the following expression to find the number of points in the multipoint:
!shape.partcount!
Any feature with two or more parts is a loop. The figure to the right shows the Calculate Field dialog. I named my field NumberOfIntersections. To display loops, I select intersections where NumberOfIntersections is greater than one.

Finding potential errors

The intersection features can help you find potential errors in the street features. For example, the map below shows an apartment complex where all the streets are named SUNRISE. This may or may not be a street name coding error, but to find these intersections and review them, you need a better way then visually scanning the table looking for issues.
To facilitate finding errors, I calculated a new field, StreetCount, containing the number of unique street names at an intersection.
The expression for Calculate Field is:
getLength(!Names!)
The getLength() routine code is as follows:
1
2
3
4
5
6
7
8
9
10
11
12
def getLength(inNames):
  # Create a list (array) by splitting the string on its delimiter
  #
  asList = inNames.split(" & ")
  # Use the set operator to create a new list with unique names
  #
  uniqueList = set(asList)
  # Return the number of entries in the set
  #
  return len(uniqueList)
To find potential issues, select records where StreetCount is not equal to Join_Count, as shown below.  Join_Count is the number of intersecting features while StreetCount is the number of unique street names. It they’re not equal, then there are multiple features with the same name at the intersection and I need to investigate further.

Ramps and limited access highways

You probably don’t want to create intersections where surface streets cross limited access highways, or where ramps connect limited access highways and surface streets.  You can avoid processing these streets by selecting just those centerlines you’re interested in, as shown in the model snippet below.
This entry was posted in Analysis & Geoprocessing and tagged , , , , , , , . Bookmark the permalink.


Wednesday, April 20, 2016

No 'Access-Control-Allow-Origin' header is present on the requested resource


Tileserver:
-----------------------------------------------------------------------------------------------------------------------
This is error message

XMLHttpRequest cannot load http://vectortile.mydomain/city/13/1411/3282.pbf. No 'Access-Control-Allow-Origin' header is present on the requested resource. Origin 'http://localhost:100' is therefore not allowed access.

I setup subdomain as
vectortile.mydomain =  http://ip:port/tileserver/tileserver.php?/index.json?/

So if i run http://vectortile.mydomain/city/13/1411/3282.pbf from my local pc, it correctly response the file.

If i run website http://localhost:100/mywebsite,  I get above error.

If tile sever run on local pc, no problem. The problem is only when I use domain host tile server.


I think the problem is subdomain with two "?"

if I do not use subdomain, instead use IP and port, will solve the problem.
------------
Not work: error message above.
set subdomain = http://ip:port/tileserver/tileserver.php?/index.json?/

tile_url = 'http://subdomain/' + 'city//{z}/{x}/{y}.pbf'
---------------------------------------------------------------------------------
remove subdomain.

Work!
tile_url = 'http://ip:port/tileserver/tileserver.php?/index.json?/city//{z}/{x}/{y}.pbf'


=======================================================================


   
    //_tileserver_base_url = 'http://vectortile.transparentgov.net/'; // NOT work,with error No 'Access-Control-Allow-Origin' header is present on the requested resource
        _tileserver_base_url = 'http://166.62.80.50:10/tileserver/tileserver.php?/index.json?/'; // must use this



    // _tileserver_url = 'http://localhost:10/tileserver/tileserver.php?/index.json?/' + _area + '/{z}/{x}/{y}.pbf';
     //   _tileserver_url = _tileserver_base_url + 'tileserver.php?/index.json?/' + _area + '/{z}/{x}/{y}.pbf';


        _tileserver_url = _tileserver_base_url + _area + '/{z}/{x}/{y}.pbf';
==============================================================

tilestream:
both ip:port or subdomain will work. 


var _tile_baseURL = 'http://166.62.80.50:8888/v2/';



            //http://tile.transparentgov.net/v2/cityadr/{z}/{x}/{y}.png
            _tile_baseURL = 'http://tile.transparentgov.net/v2/';
            // _tile_baseURL = 'http://localhost:8888/v2/cityadr/{z}/{x}/{y}.png';





                    return _tile_baseURL + _areaID + '_' + _subjectID + '/' + zoom + '/' + coord.x + '/' + coord.y + '.png';

Friday, April 15, 2016

javascript array loop

var length = arr.length;
    element = null;
for (var i = 0; i < length; i++) {
  element = arr[i];
   

  // Do something with element i.
}


new_array.push(element);


object loop through

for (var property in object) {
    if (object.hasOwnProperty(property)) {
        // do stuff
    }
}

Tuesday, April 5, 2016

PGRestAPI tessera tileserver

(((((((((((((( tessera tessera tessera tessera ))))))))))))))))))))))))))))




--------------------------------------

任何一个地方,-g meas global install,

npm install -g tessera
npm install -g mbtiles


安装地点不在上面那个地方,而是统一在

C:\Users\HU_J\AppData\Roaming\npm


现在去那个地方,

cd  C:\Users\HU_J\AppData\Roaming\npm

在那里必须按装这两个 东西,否则不能用,

npm install tilelive-mapbox tilelive-xray

安装完成,开始使用


去到有mbtiles文件的地方
cd C:\joe_hu\data_history\vector_tiles


tessera mbtiles://./City_Parcels.mbtiles




然后去

http://localhost:8080/ 就可以看到。

注意,mbtiles 是vector tile, 但是网页得到的是 png raster

=======================================================


另外一个方法是:

找一个地方,
npm install tessera tilelive-mapbox tilelive-xray


cd C:\jh1\tessera\node_modules\.bin

tessera mapbox:///mapbox.mapbox-streets-v5







(((((((((((((((((( END ((((((( tessera tessera tessera tessera ))))))))))))))))))))))))))))





(((((((((((((((((((((((((((((((((((((((((PGRestAPI PGRestAPI PGRestAPI))))))))))

MS build error, is because of missing visual studio c++ build tool

1) solution is install visual studio c++ express 2010 ( must install first and try)

2) if still has build error, read error message, if error says can't find ".......MSBuild/v140/...."
    v140 means visual studio c++ 2015,  you need to install it.

    if error says.... /v120/  means missing visual studio c++ 2013,,,,,
        v110 = 2012

v100= 2010


**************   If above still does not work, try below  ********************

I have both Visual Studio Express 2013 and Visual Studio Community 2015 installed.
I was getting  C:\Program Files (x86)\MSBuild\Microsoft.Cpp\v4.0\V120\Microsoft.Cpp.Platform.targets(64,5): error MSB8020: The build tools for v140 (Platform Toolset = 'v140') cannot be found. To build using the v140 build tools, please install v140 build tools. Alternatively, you may upgrade to the current Visual Studio tools by selecting the Project menu or right-click the solution, and then selecting "Upgrade Solution...". [G:\work\cinema\node_modules\engine.io\node_modules\bufferutil\build\bufferutil.vcxproj]
The reason was that the latest Node.js for Windows downloaded from https://nodejs.org contains an old npm v2 (and old node-gyp inside that npm v2).
I had to update Node.js'es internal npm (which also updated node-gyp):
(open console as an administrator)
cd "C:\Program Files\nodejs"
npm install npm@latest

npm config set msvs_version 2013
Now it works (seems that it's using VS 2013 for the time being)





--------------------------------------------------------------------------------

install mapnik on windows, otherwise,   run 'node app.js'  will get error saying can't find mapnik.


Before install mapnik, must install python 2.7.10 32bit X86 version,
Must set all path to this 32 bit of python.

python27 64bit Does not work.


((((((((((((((((((((((((END((((((((((((PGRestAPI PGRestAPI PGRestAPI))))))))))





Thursday, March 17, 2016

svn setup

1. install TortoiseSVN-1.8.7.25475-x64-svn-1.8.9

2.   at C:\ create a new foler SVN

       right click SVN folder, choose TortoiseSVN -- repo-browser

       URL input :  https://srv33.citycm.local/svn

        On left side panel, can see intranet1, intranet2



           export intranet1 and 2 as needed.



3.

GIS has 2 repo, you can test it and try it.













Wednesday, March 9, 2016

数据处理 processing data

1) zoning, classification:

zoning code 标题和内容,都不能有特殊字符,比如 括号(),/,都不行,空格也不行。
 但是, 连字符 - 和 _ 下划线是可以的

小数点也不行,比如 RM4.5  要改成45 或者4_5


zoning code 后面的描述,description 是可以有其它字符的


2) tabular data

excel, 标题,必须无空格,无特殊字符


================================================

shape file 处理程序,

1.   去掉没有用的field, data management > field > delete field

   tilemill 不能处理简化的时候, 没有另外保存的自身文件.
就是说哦,简化的时候,一定要另存为一个新文件, 否则,tilemill会出错.



简化是最后一步做, 之前要把zoning,做完,因为zoning很可能要该其中的内容.



1.2  有很多address的apartment, unit 是用重复的点的,parcel也是,


     这就要用 qgis, top menu - MMQGIS - modify - delete duplicate geometry 来把重复的点或parcel删除。


2.  qgis > vector > geometry  tool > simplify geometry 简化vertice,

       simplify tolerance :  2
simplify如果参数 为1 或 2 的时候simplify 出错,那就用default 参数 0.0001
1和2 减少的点数多, 0.0001减少的点数少.



     2.1  大城市downtown,有很多楼的地址点是重复的,因为每一个unit,就成为一个点,
           一个楼可能有60个unit,60个 点,这时候,就要把重复的点去掉,三番的zoning也有很     多重复的polygon,
           所以要去掉重复的geometry,
                qgis 》 mmqgis(plugin)》 modify > delete duplicate geomerty















3. GDAL OGR2GUI64.exe  把 shape file 转化成 geojson ,   projection:  "4236"

4. 处理geojson, 1去头,2去尾,3把每行的逗号去掉。

5. import to mongo.



==========================================================

city limit boundary line 处理 leaflet 版,
其它和下面的google map 版一样, 只有替换replace 部分不一样.


1   first remove all up to coordinate:

[ -81.255314684920464, 28.495430541470494 ], [ -81.244196080156513, 28.495468818460033 ],......




2.  replace ], [      >         ], \n [     注意 \n

3.  replace ,    >    ,+好多空格  ],     >     ],+好多空格

4. press alt + shift select all longitude column, cut  and paste after latitude

5, 去掉所有 逗号, 再在经度前加逗号,  最后大括号外面加逗号

6, 还要修理头尾,的几个括号

===================================================

city limit boundary line 处理 google map

1. 如果没有提供city limit,就用fire district, neighbourhood, community 等
进行 dissolve (FID) field.  check "create multipart"

               arcToolbox - data management tool - Generalization - Dissolve
               In the 'dissolved field optional '   check the FID



2. 不要这样做 因为没效率【multipart polygon进行edit vertice, 用delete vertice tool, 选中中间那些小空洞的点,删除,一选中就自动删除,只留一个外面的大polygon的边线,中间的所有点都删除。】

       另外一种办法是,用qgis, top menu - vector - geometry tool - polygon to line

变成line以后,就发现有多条line,但只有一条主要的轮廓,其它那些虽小的line都应该删除。
先选中那条主要的轮廓,然后再反选,即把主要的轮廓以外的细小的线选中。一次性删除。


3. 最后把剩下的一根外轮廓线,进行 simplify geometry ( qgis - vector -gemotry tool - simplify geometry)
    simplify如果参数 为1 或 2 的时候simplify 出错,那就用default 参数 0.0001


4. 把其变成geojson, 最后变成lat long format, 参考

4.1   first remove all up to coordinate: [[]]


4.2    3 steps: replace

[
{ lng:


]
}

, 3
, lat: 3


4.3   每一行前后处理
each line begin with:  [{
each line end should be:  } ],
last line end no comma,  should be :  }] only.


4.4 最后把这些放进js 文件里就可以了。



=============================================