So i've been pondering doing some android development. I don't really want to limit myself to android, iOS would be nice too, but I have no way to test and I'm not keen on giving apple any money just to have a play either.
For 3D stuff the choice is OpenGL ES 2.0, the obvious cross platform choice for this is
LibGDX, I've not used it before so know very little about it. When I started doing OpenGL I used raw GL to learn the basics and how the system works at it's lowest level. I built up a fairly simple scene graph engine. Then I moved on to a library that handles all those details and lets me get on with game logic. I thought I good way to get started with android/mobile 3D was to do a raw android app and the later move on to LibGDX.
Having not done any android dev my first port of call was the android docs, they even have a tutorial section on
OpenGL in android. Following this tutorial I got a simple android app up and running.
OpenGL ES 2.0 means that all the fixed function pipeline code I used to write is obsolete and I had to figure out what was needed to use the programmable pipeline. Luckily I had done a bit of work using GLSL shaders back on
DarkVoid.
For what this code does, the main difference between libgdx and pure android is the setup code. This meant that the GLES bits and pieces I learned whilst creating the pure android app was useful for the libgdx version. One thing that caught me out changing between the 2 was the way the Matrix classes work. The 2 libraries seem to do the multiplication the opposite way round. This caused me a headache as nothing was appearing on screen when I started looking at libgdx. It took a while but when I swapped over the order of the multiplication calls when calculating the model-view-projection matrix it all started working as expected.
I found it quite difficult to find any examples of doing basic 3D work in LibGDX, plenty of 2D examples with orthographic projection cameras, but for 3D and using perspective projection there was very little, hopefully some of this will help someone else out there.
Raw Android
I'm not going to go through a full android tutorial here, For those unfamiliar with android, an app starts by having a manifest file that describes an android activity. This is just a class and is used to bootstrap the app. For an opengl es app this just creates an instance of a GLSurfaceView and makes sure that the screen doesn't sleep on us. The only method we need in our activity is the onCreate implementation.
@Override
public void onCreate(Bundle savedInstanceState) {
super.onCreate(savedInstanceState);
getWindow().addFlags(WindowManager.LayoutParams.FLAG_KEEP_SCREEN_ON); //Makes sure the screen stays active
mGLView = new MyGLSurfaceView(this);
setContentView(mGLView);
}
The next piece to look at is the gl view. In the constructor for this we tell the system that we need an OpenGL ES 2 EGL context and create the GLSurfaceView.Renderer instance.
public MyGLSurfaceView(Context context){
super(context);
setEGLContextClientVersion(2); //Make sure we are using OpenGLES 2
renderer = new MyGL20Renderer();
setRenderer(renderer);
}
We also have the code that handles our
MotionEvent, we will be using this to move our camera around.
@Override
public boolean onTouchEvent(MotionEvent e) {
switch (e.getAction()) {
case MotionEvent.ACTION_MOVE:
if(e.getHistorySize()>0) {
float deltaX = e.getX() - e.getHistoricalX(e.getHistorySize()-1); //Calculate how much the mouse moved in the x axis.
float deltaY = e.getY() - e.getHistoricalY(e.getHistorySize()-1); //Calculate how much the mouse moved in the y axis.
renderer.rotateCamera(-deltaY, -deltaX); //Rotate the camera by the required amount
}
}
return true; //Return true to signify that other event handlers can also process this event.
}
The renderer code is responsible for handling the overall rendering functions of the application. There are 3 main lifecycle methods to implement here. One for when the surface is created, one for when the screen is resized and then one for doing our rendering that is called each frame. The content of these methods is mostly OpenGL ES standard calls and not android specific.
The first just sets up some of our context and creates the objects in our scene.
@Override
public void onSurfaceCreated(GL10 unused, EGLConfig config) {
GLES20.glClearColor(0f, 0f, 0f, 1.0f); //Sets our background colour to black
GLES20.glEnable(GLES20.GL_DEPTH_TEST); //Enables depth testing, so objects further from the camera are not rendered over ones that are closer
GLES20.glDepthFunc(GLES20.GL_LESS); //Sets the depth test function to use, 'less than' in our case.
triangle = new Triangle(); //We'll add a triangle and a cube to our scene.
cube = new Cube();
meshes.add(triangle); //Add the objects to the list of meshes that we will render.
meshes.add(cube);
}
The next lifecycle event that will be called is the one that handles the surface change event. This is called when directly after the creation event for the surface and any time the surface changes, like screen rotation. The important tasks here are to set the viewport and set the camera up. This includes setting up the clip planes and the projection matrix. Android has a method to do some of this for us, but it's only available in android 4, so we have to use the older method which means doing some of the maths ourselves, it's not hard though. In OpenGL there are 3 matrix that are important. The projection matrix is a matrix that describes how we will see the scene. It describes our field of view and how the perspective is calculated. The view matrix describes the location and orientation of our camera. The model matrix describes the location of the object we are rendering in the scene. Multiplying the 3 together will allow us to translate the vertex coordinates to screen space coordinates.
@Override
public void onSurfaceChanged(GL10 unused, int width, int height) {
GLES20.glViewport(0, 0, width, height); //Set the view port
float ratio = (float) width / height; //Calculate the aspect ratio
float frustumH = (float) (Math.tan(60 / 360.0f * Math.PI) * FRONT_CLIP_DISTANCE); //Calculate the height frustum
float frustumW = frustumH * ratio; //Calculate the width frustum
Matrix.frustumM(mProjMatrix, 0, -frustumW, frustumW, -frustumH, frustumH, FRONT_CLIP_DISTANCE, 3000); //Set the projection matrix from our calculated values
rotateCamera(0, 0); //Initialise our camera.
}
Next is the method that does our rendering. In this demonstration code we will also add some movement to our scene by moving and rotating the cube. We have to calculate the new position of the cube each frame so for a simple demo like this, in the rendering method is as good a place as any.
@Override
public void onDrawFrame(GL10 unused) {
GLES20.glClear(GLES20.GL_COLOR_BUFFER_BIT | GLES20.GL_DEPTH_BUFFER_BIT); //Clear the colour and depth buffers so we have a blank start each frame
long time = SystemClock.uptimeMillis() % 2000; //Calculate the amount to move/rotate our objects
float deltaTime = time/2000f;
float offset = (float) Math.sin(deltaTime * 2 * Math.PI);
cube.setPosition(new float[] {offset * 3f,0,-3}); //Move our cube
cube.setRotation(new float[] {1,0,0,deltaTime * 360}); //Rotate our cube
for(Mesh mesh: meshes) {
mesh.draw(mVMatrix, mProjMatrix); //Render the items in the list passing in the view matrix and the projection matrix
}
}
The view matrix that we used is set when the rotateCamera method is called. For the demo we want a control method that is simple to use so we restrict the orientations the camera can be placed in. The camera will always be right side up so we restrict motion in the x axis to +/- 90 degrees. For simplicity we also keep the y rotation to +/-180, but we wrap this when we get to one limit or the other. We apply the transform first and then apply the rotation. The translation of the camera is so that we can look at the origin of the scene (0,0,0). Then we apply the rotation so that the camera is cameraDistance away in which ever direction we intended to be looking. The camera location is then a translation in the camera local coordinate system.
public void rotateCamera(float xAngle, float yAngle) {
Matrix.setIdentityM(mVMatrix, 0);
Matrix.translateM(mVMatrix, 0, 0, 0, -cameraDistance);
cameraYRotation = cameraYRotation + yAngle;
if(cameraYRotation>180) cameraYRotation = cameraYRotation - 180;
if(cameraYRotation < -180) cameraYRotation = cameraYRotation + 180;
cameraXRotation = cameraXRotation + xAngle;
if(cameraXRotation > 90) cameraXRotation = 90;
if(cameraXRotation < -90) cameraXRotation = -90;
Matrix.rotateM(mVMatrix, 0, -cameraXRotation, 1, 0, 0);
Matrix.rotateM(mVMatrix, 0, -cameraYRotation, 0, 1, 0);
}
That is all the scene setup and control done. Now we can concentrate on the objects in the scene. All OpenGL objects are triangles, you stick multiple triangles together to make more complex shapes, with enough, you can make the shapes that build up a sphere, or a spaceship or a player character. These triangles have attributes that give them more than just an abstract presence. They have vertices that describe the location of each point. Each vertex can have a colour, a texture coordinate for applying a texture, a normal, which describes the direction the surface is facing at that vertex which is used for lighting effect, and the list of possible attributes goes on. In OpenGL, the fixed function pipeline handles this data and figures out how to deal with it. With the programmable pipeline we have to process the data ourselves in the vertex and fragment shaders. The vertex shader is called for each vertex in the triangle and figures out where on the screen it will be rendered, the fragment shader is caller per pixel and is responsible for figuring out what colour each pixel between the vertices should be. This means we can do various things in the fragment shader, like a simple colour applied over the whole object, or texturing, per pixel lighting, and more complicated things like bump mapping, normal mapping etc. For this demo we'll just do a simple colour over the whole model.
The shapes we used above are a simple triangle 2D and a 3D cube. We'll be building them out of triangles, in the case of the triangle it's obviously a single triangle, in the case of the cube we need the 6 square faces, each being 2 triangles. One way to do this in OpenGL is to specify the coordinates of each vertex in the cube (8 of them) once, and then use an index in to an array of these vertices. Otherwise we would have to list all vertices for all all triangles, all 36 of them. As these are passed to the shaders every single frame we want to reduce this, using indexed triangle strips reduces this number dramatically.
The only difference in function between the triangle and the cube is the number of vertecies, the colour we intend to use and the indexes used to create the shapes so we can create a common class, I've called this a mesh.
Here is the constructor for our mesh
public Mesh(float[] vertecies, float[] colour, short[] vertexOrder, boolean isTwoSided) {
numVertecies = vertexOrder.length;
this.isTwoSided = isTwoSided;
this.colour = colour;
vertexBuffer = ByteBuffer.allocateDirect(BYTES_PER_FLOAT * vertecies.length).order(ByteOrder.nativeOrder()).asFloatBuffer();
vertexOrderBuffer = ByteBuffer.allocateDirect(BYTES_PER_SHORT * vertexOrder.length).order(ByteOrder.nativeOrder()).asShortBuffer();
vertexBuffer.put(vertecies).position(0);
vertexOrderBuffer.put(vertexOrder).position(0);
int vertexShader = Mesh.loadShader(GLES20.GL_VERTEX_SHADER, vertexShaderCode);
int fragmentShader = Mesh.loadShader(GLES20.GL_FRAGMENT_SHADER, fragmentShaderCode);
shaderProgram = GLES20.glCreateProgram(); //Create the shader program
GLES20.glAttachShader(shaderProgram, vertexShader); //Add the vertex shader
GLES20.glAttachShader(shaderProgram, fragmentShader); //Add the fragment shader
GLES20.glLinkProgram(shaderProgram); //Compile the program
}
There is one final parameter here, isTwoSided, this is used to tell our code wether the shapes should be drawn to be visible from all sides. In the case of the cube this would mean from the inside as well as outside of the cube. In the case of the triangle it's from the front as well as the back.
Most of this code handles setting up the 2 in memory buffers we will use to pass the list of vertices and the indexes to the shaders. Direct ByteBuffer are a very efficient way of doing this as they are raw blocks of memory that the bytes that make the floats that make the vertices is placed directly into.
Next up is the code that sets up the context for displaying this mesh and passing the attributes to the shader program. As mentioned before we need the view matrix and projection matrix at this point to be able to pass to the shader program that will combine them with the model matrix and the vertex position to transform it to screen space.
public void draw(float[] mVMatrix, float[] mProjMatrix) {
if(isTwoSided) {
GLES20.glDisable(GLES20.GL_CULL_FACE);
} else {
GLES20.glEnable(GLES20.GL_CULL_FACE);
}
GLES20.glUseProgram(shaderProgram);
int mPositionHandle = GLES20.glGetAttribLocation(shaderProgram, "vPosition");
GLES20.glEnableVertexAttribArray(mPositionHandle);
GLES20.glVertexAttribPointer(mPositionHandle, DIMENSIONS,
GLES20.GL_FLOAT, false,
VERTEX_STRIDE, vertexBuffer);
int mColorHandle = GLES20.glGetUniformLocation(shaderProgram, "vColor");
GLES20.glUniform4fv(mColorHandle, 1, colour, 0);
float[] worldPositionMatrix = new float[16];
Matrix.setIdentityM(worldPositionMatrix , 0);
Matrix.translateM(worldPositionMatrix, 0, worldPosition[0], worldPosition[1], worldPosition[2]);
Matrix.rotateM(worldPositionMatrix, 0, worldRotation[3], worldRotation[0], worldRotation[1], worldRotation[2]);
float[] mvpMatrix = new float[16];
Matrix.multiplyMM(mvpMatrix, 0, mVMatrix, 0, worldPositionMatrix, 0);
Matrix.multiplyMM(mvpMatrix, 0, mProjMatrix, 0, mvpMatrix, 0);
int mVPMatrixHandle = GLES20.glGetUniformLocation(shaderProgram, "uMVPMatrix");
GLES20.glUniformMatrix4fv(mVPMatrixHandle, 1, false, mvpMatrix, 0);
GLES20.glDrawElements(GLES20.GL_TRIANGLE_STRIP, numVertecies,
GLES20.GL_UNSIGNED_SHORT, vertexOrderBuffer);
GLES20.glDisableVertexAttribArray(mPositionHandle);
}
First we set back face culling based on the 2 sided property of this mesh. Then we tell GL that we want to use the shader. Next, we find the location of each of the attributes for the shader, then we can set the attribute value to that location. So we set up the list of vertices, the colour and the mvpMatrix. Then we tell GL to draw the triangle strip. The mvpMatrix is the matrix that describes how to convert the vertex coords in model space to screen space, it is the combination of the projection matrix, the view matrix and the model matrix. We only need to calculate this once per frame per model so it's worth doing in our code and then passing to the shader each frame rather than calculating it in the shader program for each vertex.
So far, we have yet to get anything on screen. The last part of puzzle is the shader itself. These are really simple for our demo application. The vertex shader doesn't do much, it multiplies the vertex being calculated by the mvpMatrix so that it's in screen space. That's it.
uniform mat4 uMVPMatrix;
attribute vec4 vPosition;
void main() {
gl_Position = uMVPMatrix * vPosition;
}
Simple. You can see that we have 2 parameters, the mvpMatrix, passed via a uniform, which means it's the same for each execution of the shader (for this frame), the vPosition attribute changes as each vertex is processed.
The fragment shader is also very simple, it just returns the colour for the pixel, as this shader just returns a single colour for the whole object it really simply just returns that colour.
uniform vec4 vColor;
void main() {
gl_FragColor = vColor;
}
For the fragment shader we also have a uniform attribute, this time it's the colour for the shape, it's the same colour for each pixel that is rendered so it's a uniform param.
All we need now is the cube and triangles themselves, these are just extensions on the mesh that pass the correct vertices, indexes, colour and two sidedness of the mesh.
All the code for this example is in a git repository on
github.LibGDX
There are several advantages of using LibGDX. The biggest being the cross platform nature. I started by using the maven archetype to generate the projects. This gives us 5 projects, 4 backend projects that cover the platform specific life cycle bits for android, iOS, desktop and GWT. The 5th project is the core project, as much as possible goes in this project, all the rendering functionality, the shapes, the input handling can all go here. I'm only going to cover android and desktop for now.
Android
The android project contains some similar parts to the previous project, the xml descriptor and the bootstrap activity. The only method this needs is the onCreate method. Everything else will be passed through to our core project by the LibGDX code.
Here we set up the application config, in this case we need to specify GL ES 2, wake lock (to keep the screen alive), disable the compass and accelerometer (we don't need them) and then set the number of samples to 2, this is for mutlisampling, this will produce smooth lines on the screen rather than jaggy lines.
@Override
public void onCreate(Bundle savedInstanceState) {
super.onCreate(savedInstanceState);
AndroidApplicationConfiguration config = new AndroidApplicationConfiguration();
config.useGL20 = true;
config.useWakelock = true;
config.useAccelerometer = false;
config.useCompass = false;
config.numSamples = 2;
initialize(new OpenGLTest(), config);
}
Desktop
For the desktop project you just need to create a Main class. Again, all we need to do is set up the configuration for the application and then create our core app.
public static void main(String[] args) {
LwjglApplicationConfiguration config = new LwjglApplicationConfiguration();
config.useGL20 = true;
config.title = "LibGDX Test";
config.samples = 2;
new LwjglApplication(new OpenGLTest(), config);
}
Core
Now we can focus on our application. We need a few lifecycle events again, create, resize and render. They format should be familiar if you have read the rest of this piece, the methods are here to show the similarities and differences
@Override
public void create () {
Gdx.app.setLogLevel(Application.LOG_DEBUG);
Gdx.app.debug("init", "In create");
cube = new Cube();
renderables.add(cube);
renderables.add(new Triangle());
camera = new PerspectiveCamera(67, 1, 1);
camera.near = 1;
camera.far = 3000;
Gdx.input.setInputProcessor(this);
Gdx.gl20.glClearColor(0, 0, 0, 1);
Gdx.gl20.glEnable(GL20.GL_DEPTH_TEST);
Gdx.gl20.glDepthFunc(GL20.GL_LESS);
fpsLogger = new FPSLogger();
}
Set up the objects in the scene and the basic camera properties including creating the LibGDX perspective camera. This does the maths for us that we had to do in pure android to create the projection matrix.
@Override
public void resize (int width, int height) {
Gdx.app.debug("init", "In resize");
camera.viewportWidth = width;
camera.viewportHeight = height;
rotateCamera(0,0);
}
Set up the camera items that change when the screen is resized
@Override
public void render () {
Gdx.gl20.glClear(GL20.GL_COLOR_BUFFER_BIT | GL20.GL_DEPTH_BUFFER_BIT);
float animationRatio = (TimeUtils.millis() % 2000)/2000.0f;
float offset = MathUtils.sin(animationRatio * (MathUtils.PI2));
cube.setPosition(offset * 3f,0,-3);
cube.setRotation(animationRatio * 360, 1, 0, 0);
for(Renderable renderable : renderables) {
renderable.render(camera.view, camera.projection);
}
}
Update the location and rotation of the cube and then run through the list of renderable objects. As LibGDX already has the concept of a mesh I added a Renderable interface that Shapes implement.
To managed the camera rotation we also have to implement the InputProcessor interface, the only method we need an implementation of is touchDragged. This is enough to enable us to rotate the camera around like we did in the pure android version.
@Override
public boolean touchDragged(int screenX, int screenY, int pointer) {
rotateCamera(-Gdx.app.getInput().getDeltaY(pointer),-Gdx.app.getInput().getDeltaX(pointer));
return false;
}
One thing that is missing from the pure android version is managing of the objects we create, specifically when they need to be destroyed. LibGDX makes this clear and easy, any object that has a destroy method should have that method called when your application goes away. All our objects are attached to a Renderable, so I added the destroy method there and then we can just run over them and destroy them.
@Override
public void dispose () {
for(Renderable renderable : renderables) {
renderable.dispose();
}
}
We also need our camera rotation method again
private void rotateCamera(float xRot, float yRot) {
cameraXRot += xRot;
cameraYRot += yRot;
if(cameraYRot>180) cameraYRot = cameraYRot - 360;
if(cameraYRot < -180) cameraYRot = cameraYRot + 360;
if(cameraXRot > 90) cameraXRot = 90;
if(cameraXRot < -90) cameraXRot = -90;
camera.position.set(0,0,7.1f);
camera.position.rotate(cameraXRot, 1, 0, 0);
camera.position.rotate(cameraYRot, 0, 1, 0);
camera.lookAt(0, 0, 0);
camera.update();
}
As LibGDX has it's own camera object we need to manipulate that object rather than directly dealing with the matrices. This sets up the view matrix that we had to do ourselves in pure android.
In the Shape class we only need to initialise the object and add a render method. The shader code is exactly the same code as we used on the pure android example.
public Shape(float[] vertecies, short[] indecies, boolean twoSided, float[] colour) {
mesh = new Mesh(true, vertecies.length, indecies.length, new VertexAttribute(Usage.Position, 3, "vPosition"));
mesh.setVertices(vertecies);
mesh.setIndices(indecies);
shader = new ShaderProgram(vertexShaderCode, fragmentShaderCode);
if(!shader.isCompiled()) {
Gdx.app.error("shader", "Failed to compile shader");
Gdx.app.exit();
}
mesh.bind(shader);
this.twoSided = twoSided;
this.colour = colour;
}
We create the LibGDX mesh, pass the vertices and indexes down to the mesh and then create the shader program. In LibGDX we then bind the shader to the mesh.
public void render(Matrix4 viewMatrix, Matrix4 projectionMatrix) {
if(twoSided) {
Gdx.gl20.glDisable(GL20.GL_CULL_FACE);
} else {
Gdx.gl20.glEnable(GL20.GL_CULL_FACE);
}
position.idt();
position.translate(translation);
position.rotate(rotation[1], rotation[2], rotation[3], rotation[0]);
mvpMatrix.set(projectionMatrix).mul(viewMatrix).mul(position);
shader.begin();
shader.setUniform4fv("vColor", colour, 0, 4);
shader.setUniformMatrix("uMVPMatrix", mvpMatrix, false);
mesh.render(shader, GL20.GL_TRIANGLE_STRIP);
shader.end();
}
In our render method we need to enable or disable back face culling again, then set the model matrix for this object. Then as before we calculate the model-view-projection matrix to pass to the shader program. Setting the attributes on the shader is handled by LibGDX for us, we can reference them by name and we don't have to worry about creating ByteBuffers to manually manage the memory.
The dispose method just handles disposing the shader and the mesh we used for rendering the Shape.
@Override
public void dispose() {
shader.dispose();
mesh.dispose();
}
Our Triangle and Cube Shapes can now be created simply like the android version
public Triangle() {
super(new float[] {0.0f, 0.622008459f, 0.0f,
-0.5f, -0.311004243f, 0.0f,
0.5f, -0.311004243f, 0.0f }, new short[] { 0, 1, 2 }, true, new float[]{0,1,0,1});
}
public Cube() {
super(new float[] { 1f, 1f, 1f, -1f, 1f, 1f, -1f, -1f, 1f,
1f, -1f, 1f, 1f, 1f, -1f, -1f, 1f, -1f, -1f, -1f, -1f, 1f, -1f,
-1f }, new short[] { 0, 1, 3, 2, 6, 1, 5, 0, 4, 3, 7, 6, 4, 5 }, false, new float[] {1,0,0,1});
}
The code for the LibGDX version is also available in github.
That's all for now. I hope this article proves useful to someone out there.