This will select the whole Geometry block, and you'll know where to look for the appropriate VertexAttributeList. There are items.
Each item has 3 component. But what component? Integers, Floats? Encoded on 16 bit, or 32 bit? Since the word "stride" doesn't appear anywhere in the file and all attributes start at different offsets, we can conclude that Sketchfab viewer or OSG. JS, should I say uses the data layout that I called "A" above. In this case, the stride is the size of the item. Regarding the component type, the mention to Int32Array suggest that they are 32 bit signed integers.
This really doesn't look like a truck. As it turns out, I underestimated the "Encoding": "varint" line. This is an encoding used to represent series of integers in a more compact way than just concatenating them.
Well, Marmoset vertex attributes were not compressed; here they are compressed twice! We can find details about varints here. So I made a little varint. Still not working.
I performed some basic checks with my varint decoder, and in particular when I decode the TexCoord attribute, the decoding process stops right before the offset at which the Vertex buffer starts, so this must be right. I did not want to do that at first, but I ended up reading the viewer's source code. I found the varing decoding function line and it matches my implementation. I found bounding box correction, I read the vertex shader.
Decoding the index buffer seems complicated as well. The index buffer includes a header, and my attempts to get something out of it were unsuccessful. So, shall we give up? No, I think it is time to step back a bit.
We focused on trying to decode the data as it comes from the server, when it first reaches the client's computer. But instead of mimicking the data decoding process performed by the viewer, couldn't we let it do its job and extract the data after this is done?
Intercepting the data between the server and the viewer was convenient, because the network monitor gives us direct access to all the resources.
But once in the memory of the JavaScript VM, it is not so easy to to extract it, even with the Memory Inspector provided by the browser's tools.
Furthermore, some of the data handling is done in the vertex shader, so on the GPU. This will hence never be accessible from the browser. This is where RenderDoc comes in. Think of it as the browser's developer tool, but for monitoring the GPU. RenderDoc records all the calls to the graphic API and their arguments, so that it is able to fully replay it and inspect it step by step.
NB RenderDoc is not intended to be used as a hacking tool, and is not supported for that. It is a GPU debugger, but as a side effect of being a good debugger, it gives us access to the data we need.
Before trying more complicated stuff, it is good practice to start with a simple example. I will use for this an OpenGL 4. Its render function is very minimal:. We build this, then launch it with RenderDoc. The application will run with an overlay with information about the graphical API in use OpenGL, here and the frame number. As it says, press F12 to capture a frame. In the RenderDoc window, the capture appears, and you can double click to open it. You will then see the list of rendering "events" in the Event Browser window.
We recognize our calls to glClear and glDrawArrays. The buffer swap is done in the main loop, by glfwSwapBuffers. And what about the glUseProgram for instance?
For some reason, only calls that affect the framebuffer are displayed here, but you can see the other calls when selecting a line in the Event Browser. The other key element of RenderDoc is the Pipeline State panel. It describes the state of the GPU's rendering pipeline during the draw call.
The first step, VTX, describe the draw call input, so the vertex attributes, and the topology. Exactly what we were looking for! In this example, there are two attributes, vPosition and vColor. There is no index buffer, because glDrawArrays is not an indexed draw call. It assumes that the index buffer is [0, 1, 2, The right arrow in the vertex attributes array leads us to the Mesh Output panel, and there is again good news here.
This panel shows all the values of the input vertex attributes, but also the output of the vertex shader! And we can easily save them to disc. We now know how RenderDoc works, and we have an example to experiment with set up. But let's head back to our original problem. We need to capture frames in the browser. Logically, we try to launch the browser from RenderDoc, as we did with our little test program.
Unfortunately, this does not work, neither will it work with other browsers. I am not sure exactly why, but the use of process forks and some other advanced mechanism must loose RenderDoc. Since RenderDoc needs to get attached before any graphical operation is initiated, we will use the --gpu-startup-dialog argument available in Chrome.
It opens a popup telling on which GPU chrome is running, and does so before initializing the graphics. So we can launch chrome, attach RenderDoc, and only then press the Ok button on the popup. For convenience, I created a shortcut with the target:. It is important that after the injection, and after you validated the popup, the status box in RenderDoc shows something different than None in front of "API:". Now, open the Sketchfab page of your choice, and take a capture using F To ensure that the capture will include the part about the 2D model, turn the 3D view at the same time.
This way, the GPU will have to recompute it. This time, the render events list is much longer than in our example. This may vary, but I have calls to the graphics API recorded, organized in many passes. If you have significantly less events, or just a single render pass, this might be that you captured a compositing pass, drawing the UI of the browser instead of the viewer events.
There are many draw calls. Some render the same list of attributes, with the same index buffer. Typically, the first pass, or the two first passes, are rendered from the position of the light, and used to compute the shadow maps. Track the draw calls that use all the textures, using the Texture Viewer panel. In my case, it is in pass 3. My pass 3 has 5 draw calls.
The last one has only 4 points, it is the ground plane. The previous one are the two parts of the mesh, with for each one a triangle strip followed by a triangle list to fill the holes. This matches the description found in the JSON file. Note that the vertices in the CSV are sorted, and potentially repeated, according to the index buffer, so you don't have to worry about this. This is a very basic algorithmic exercise: given a list of indices [1, 2, 3, 4, 5, 6, 7, Note that the second triplet is 3, 2, 4 and not 2, 3, 4 in order to preserve the face normal.
Every odd triplet must be reversed like this. Once again, I don't want to make it easy for anybody to download models from Sketchfab, so I won't provide any full-featured script. I will have less concern regarding Google Maps. If you are publishing content on Sketchfab, you might be a bit worried. But don't worry too much. It is still a lot of work, and skilled work, to steal your models, so in many cases if you sell your model at a reasonable price it will remain more expensive for a company to steal than to buy.
And it gets even more complicated when your model is animated, or uses advanced rendering effects. If you really freak out, add watermarks directly on the textures of the model. Of course it does not protect the geometry, but what is a geometry without its texture? Especially if it is a low poly on which a high definition model has been baked. If you sell point clouds from 3D scans, don't include all the points in the WebGL preview, or maybe show the true density only on one part of it.
If you sell triangles, cut you mesh into many little meshes, it'll get longer to retrieve it. I don't know, get inventive, or trust people. Anyway, it remains illegal to use your stolen models, which adds another cost to the thief. Which data bottleneck should we focus on? The main difference between Google Maps and the previous examples is that Google Maps never loads the full 3D model of the whole world , obviously.
Instead, it uses a powerful Level of Detail LoD mechanism and streams the geometry at a resolution depending on the distance to the view point. There may be more than 10 levels of details covering a well scanned area.
Ideally, we would prefer to find how to decode the queries, and even how to emit them. This way, we would be able to query the higher resolution models for a whole region.
This never happens in practice, because of the LoD mechanism, so intercepting the data when it reaches the GPU does not allow for this. At least, not from a single capture. When one moves around in the Google Map's 3D view, data comes in a lot of little queries encoded using protocol buffer and raw octet streams, which are very dense and so hard to decode without the format information. It does not contain meta information like the data keys of a JSON file.
So it is capital to hand on the source that decodes this data. But if you got lost while analyzing Sketchfab viewer's code, you'll never come back from Google's code exploration.
Google uses a lot of optimizations that make their code totally obscure. Long story short, I have not been able to decode Google Maps packets, but for those of you who would feel giving it a shot, here are some entry points to the code.
We can spot a linear algebra library with lines such as a. XMLHttpRequest is wrapped in a function, so a little harder to track. Looking for typed arrays like Uint8Array gives more angles.
No reason to give up, we've got our RenderDoc hammer and all this 3D in Google Maps story looks like a nail. Same process than for Sketchfab: start Chrome with the --gpu-startup-dialog flag, inject RenderDoc into the process, browse to your favorite location in Google Maps, and capture a frame. Again, make sure you move in the viewport while capturing the frame. For those of you who want to play with us but could not get RenderDoc injected into Chrome, I let you download a capture of a touristic spot you might know.
This can be openned and explored with RenderDoc even if it has been captured on another computer. For the record, I used RenderDoc v1. So, here we are, all together with a frame captured from Google Maps. There is no less than calls to the graphics API in my example! If you have only one color pass, it means that you were not moving while capturing the frame. In order to figure out what the draw calls do, open on the right-hand side the Texture Viewer panel.
This shows the framebuffer the output as it was after the selected call. Then browser the calls in the Event Browser , you will see the image progressively building:. The pass 1 draws the map with the streets and their names, that will be overlayed on top of the 3D. The pass 2 draws all the 3D, as well as the little shops and restaurants indicators. Then the pass 3 composites the UI elements on top of it and turns it upside down, dunno why. That last pass is the responsibility of the browser, it is not provoked by Google Maps' code.
But, hey, there are draw calls in pass Actually "only" 57 of them are about the 3D, but it is still a lot of work, plus they all use a different texture that we can save from the Texture Viewer. And it does not stop here: many chunks of the map will be in lower level of detail, so to correctly capture an area, we might need several captures. Still not giving up!
It used to be a little tricky to use, but since the version 1. There are two APIs: qrenderdoc and renderdoc. The former starts with a 'q' for Qt because it gives acces to the graphical UI elements of RenderDoc.
It can be a way to mimic the click on the save button. But the second API is more interesting: it gives access to the captures at a lower level, more robust to changes in the UI. You don't need to start RenderDoc to use the renderdoc Python module. This is great, because it enables us to call it directly from a Blender plugin, for instance. There is just a little boilerplate: manipulating the low level API may conflict with the UI, we must wrap all our instructions in a function and call it with BlockInvoke :.
The pyrenderdoc variable is set by the interactive shell to the current CaptureContext. People also like. Walk the World Free. MapCortex - Free Edition Free. Galaxy Explorer Free. ISS Finder Free. Space Museum - Solar System Free. Inception VR Free. Mont Blanc. What's new in this version Version 3. Additional information Published by Ivan Fuligni. Published by Ivan Fuligni.
Developed by Ivan Fuligni. Approximate size Age rating For all ages. This app can Use your location Use your microphone Access your Internet connection Access your home or work networks Access your Internet connection and act as a server. Programmatically access your 3D Objects. The cookie is used to store the user consent for the cookies in the category "Other.
The cookie is used to store the user consent for the cookies in the category "Performance". It does not store any personal data. Functional Functional. Functional cookies help to perform certain functionalities like sharing the content of the website on social media platforms, collect feedbacks, and other third-party features.
Performance Performance. Performance cookies are used to understand and analyze the key performance indexes of the website which helps in delivering a better user experience for the visitors.
Analytics Analytics. Analytical cookies are used to understand how visitors interact with the website. These cookies help provide information on metrics the number of visitors, bounce rate, traffic source, etc.
Advertisement Advertisement. Advertisement cookies are used to provide visitors with relevant ads and marketing campaigns. These cookies track visitors across websites and collect information to provide customized ads.
0コメント