Sandbox, meet Playground. SceneKit & SpriteKit in Swift Playgrounds for iPad

Exploring iOS 10 (updated for beta 4)

In the Platforms State of the Union session at WWDC, it was announced that the Playgrounds app would place “the entire iOS SDK at your fingertips.” In theory, that makes it an incredibly powerful tool. At present, the only other iOS App that allows you to “code native” (ie using Apple’s extensive API library), is the insanely great Continuous, a C# / F# IDE for iOS that enables you to use Apple’s APIs via the cross-platform Xamarin environment (which was recently purchased by Microsoft). Xamarin offers C# wrappers for the whole of the Apple SDK, and even manages to do this in sync with Apple’s own release cycles (Xamarin support for iOS 9 released the same day iOS 9 did). Continuous released just a few weeks ago, not long after the beta of iOS 10 launched. No sooner had I got used to coding native on my iPad with the new Swift Playgrounds, than a second app came along with a similar feature set.

Apple has two great high-level frameworks targeted at game development. SpriteKit for 2D, and SceneKit for 3D. Both come with physics engines built-in (Box2D and Bullet 3D), and both are tonnes of fun to play with. Their physics engines give both libraries something of a sandbox feel to them, a sandbox that would be great fit for Playgrounds.

An increasing amount of research points to the importance of games in how we learn and discover. One offshoot of this realization is that gamification of non-game apps—spanning the fields of education, health, productivity, and customer engagement—is a fast expanding field. Teaching game-coding on the other hand can be thought of as the flip-side of gamification: turning a game into a learning experience, rather than the other way around. I bet there are a lot of coders who first learnt to code by creating games, and the lessons it teaches, in how to handle mutability of state, complex interactions between diverse actors, the user’s experiences of discovery and reward, not to mention performance concerns, have extremely broad applications.

SceneKit

Unfortunately, at the time of writing (beta 3), Apple’s claim to support the full iOS library falls down a little when it comes to SceneKit and SpriteKit, with important parts of these libraries currently MIA. Worst hit is SceneKit. If you attempt to set the position of a node, the Playground execution halts with an alert telling you to “check your code for errors”. The same thing occurs if you try to add a light, or a camera. I’ve been submitting bug reports, and you should too.

Update beta 4, 3 August 2016

With the arrival of iOS 10 beta 4, a lot of the problems with SceneKit and SpriteKit that I described in the earlier version of this post have been fixed. I’ve updated the code samples below and in the repository accordingly.

As I described in the previous post on UIKit in Playgrounds, in order to have a continuously running playground that can deal with inputs such as touch events, you need to assign a view to the Playgounds live view, in this case an SCNView:

import SceneKit
import PlaygroundSupport
let scene = SCNScene()

// set up your scene here

//In Xcode on the Mac, you have to supply a size for the view. On iPad, this isn't necessary
let view = SCNView() //iPad version
//let view = SCNView(frame: CGRect(x: 0, y: 0, width: 400, height: 600)) //Xcode version
view.allowsCameraControl = true
view.autoenablesDefaultLighting = true
view.scene = scene
PlaygroundPage.current.liveView = view

The code in the repo applies a metallic texture to the front of the SCNText geometry using the new physics-based rendering (PBR) material properties, introduced with iOS 10. To add some noise to the texture I use SpriteKit’s SKTexture noiseWithSmoothness method, as I wanted a quick one-liner texture. You could really go to town if you used the new GameplayKit noise methods though.

Note that if you run the playground in Xcode on the Mac, these PBR properties won’t appear, I think because PBR requires Metal, and the iOS Simulator in Xcode is backed by OpenGL, not Metal. If you did want to experiment with PBR in a Mac-based Xcode playground, you’d need to update to MacOS Sierra, and set the playground up as a MacOS playground. Playgrounds in Xcode should be set up where possible as OS X/MacOS playgrounds anyway, as the iOS Simulator runs quite slowly.

For iPad coders who’d like to explore coding the iOS SDK in C# or F#, I’m happy to say that SceneKit also runs beautifully in Continuous.

Emoji assets

SpriteKit

With SpriteKit the situation is a little different. Currently the only type of content-displaying node that actually works in iPad Playgrounds is SKLabelNode, the node that outputs text. SKSpriteNode and SKShapeNode produce the “check your code for errors” alert, as does defining an SKTexture (although strangely you can assign a texture’s .cgImage). Aside from not actually being able to display anything but text (and that is a huge limitation), the rest of the SpriteKit API seems to be working in iPad Playgrounds.

Thankfully, beta 4 has fixed these issues, so it’s now possible to have image-based sprites. Here’s a simple demo of dragging and dropping a SpriteKit node. It subclasses SKScene in order to access the override methods for when touches begin, move, and end. If you wanted to implement more sophisticated UIGestureRecognizers for pinch-to-zoom and so on, those can be attached to the scene’s view property.


import SpriteKit
import PlaygroundSupport

func - (left: CGPoint, right: CGPoint) -> CGPoint {
    return CGPoint(x: left.x - right.x, y: left.y - right.y)
}

func + (left: CGPoint, right: CGPoint) -> CGPoint {
    return CGPoint(x: left.x + right.x, y: left.y + right.y)
}

func += ( left: inout CGPoint, right: CGPoint) {
    left = left + right
}

let degree = CGFloat(M_PI_2) / 90

class GameScene: SKScene {
    var selectedNode: SKNode?
    var shakeAction: SKAction?
    
    override init(size: CGSize) {
        
        let text = SKLabelNode(text: "Drag me 🤖")
        text.fontColor = #colorLiteral(red: 0.854901969432831, green: 0.250980406999588, blue: 0.47843137383461, alpha: 1.0)
        text.position = CGPoint(x: size.width / 2, y: size.height/2)
        let sprite = SKSpriteNode(color: #colorLiteral(red: 0.854901969432831, green: 0.250980406999588, blue: 0.47843137383461, alpha: 1.0), size: CGSize(width: 30, height: 30))
        sprite.position = CGPoint(x: 100, y: 100)
        super.init(size: size)
        makeShakeAction()
        addChild(text)
        addChild(sprite)
    }
    
    func makeShakeAction(){
        var sequence = [SKAction]()
        for _ in 0..<10 {
            let shake = CGFloat(drand48() * 2) + 1
            let shake2 = CGFloat(drand48() * 2) + 1
            let duration = 0.08 // + (drand48() * 0.14)
            let antiClockwise = SKAction.group([
                SKAction.rotate(byAngle: degree * shake, duration: duration),
                SKAction.moveBy(x: shake, y: shake2, duration: duration)
            ])
            let clockWise = SKAction.group([
                SKAction.rotate(byAngle: degree * shake * -2, duration: duration * 2),
                SKAction.moveBy(x: shake * -2, y: shake2 * -2, duration: duration * 2)
            ])
            sequence += [antiClockwise, clockWise, antiClockwise]
        }
        
        
        shakeAction = SKAction.repeatForever(SKAction.sequence(sequence))
        
    }

    required init?(coder aDecoder: NSCoder) {
        fatalError("init(coder:) has not been implemented")
    }
    
    override func touchesBegan(_ touches: Set<UITouch>, with event: UIEvent?) {
        let touch = touches.first
        guard let positionInScene = touch?.location(in: self) else {return}
        
        if let touchedNode = self.nodes(at: positionInScene).first {
            selectedNode = touchedNode
            selectedNode?.run(shakeAction!, withKey: "shake")
        }
        
    }
    
    override func touchesMoved(_ touches: Set<UITouch>, with event: UIEvent?) {
        guard let touch = touches.first else {return}
        let translationInScene = touch.location(in: self) - touch.previousLocation(in: self)
        if let selected = selectedNode {
            selected.position += translationInScene
        }
    }
    
    override func touchesEnded(_ touches: Set<UITouch>, with event: UIEvent?) {
        if selectedNode != nil {
            selectedNode?.removeAction(forKey: "shake")
            selectedNode = nil
        }
    }
}


let frame = CGRect(x: 0, y: 0, width: 400, height: 600)
let view = SKView(frame: frame) 
let scene = GameScene(size: frame.size)
view.presentScene(scene)
PlaygroundPage.current.liveView = view

Code is also available in this repo.

I’m delighted that with the arrival of iOS10 dev beta 4 SceneKit and SpriteKit are now useable in iPad Playgrounds. I’m still exploring the extent to which different kinds of assets are accessible on the iPad. I imagine though that anything that the SceneKit.ModelIO module can process is useable. So far I’ve managed to import a wavefront .obj 3D model from my Dropbox into SceneKit on iPad as a file literal. I’m looking forward to trying some .dae files. In the playground’s “More” menu (the three dots icon in the right of the nav bar), if you select Advanced > View Auxiliary Source Files you can view which files you’ve attached to the project. You don’t see a rendering of the model like you would in the Mac’s Finder, but, hey, who knows what’s around the corner in future versions of Playgrounds?

Built with Jekyll      © Salt Pig Media