Display

This section covers all of the RPCs that relate to manipulation of the customizable interactive display on the mobile device. The device display is a large rectangular region called the canvas.

Locations on the canvas are defined by x and y coordinates that range from [0, 100]. These coordinates are relative to the size of the display. Thus, a button at position [25, 25] and size [50, 50] will take up a fourth of the screen (half the width and half the height), and will be centered in the canvas.

Many controls take optional parameters, which are specified as a list of pairs (lists of size 2), where each pair is the name of an option and the desired value. For instance, most controls take one or more optional parameters to control the color of the display. You can obtain color codes from PhoneIoT.getColor().

RPCS

PhoneIoT.addButton(device: Device, x: Position, y: Position, width: Size, height: Size, text: String?, options: Object?)

Adds a button to the display with the given position and size. If not specified, the default text for a button is empty, which can be used to just make a colored, unlabeled button. The text can be modified later via PhoneIoT.setText().

Arguments:

  • device: Device info - id of the device

  • x: Position info - X position of the top left corner of the button (percentage)

  • y: Position info - Y position of the top left corner of the button (percentage)

  • width: Size info - Width of the button (percentage)

  • height: Size info - Height of the button (percentage)

  • text: String? info - text to display on the button (default empty)

  • options: Object? info - Additional options

    • id: String? info - The id to use for the control. If not specified, a new one will be automatically generated.
    • event: String? info - The name of a message type to be sent each time the button is pressed. You must call PhoneIoT.listenToGUI() to actually receive these messages. If not specified, no event is sent. Message fields: id.
    • style: ButtonStyle? info - The display style of the button on the screen. This can be rectangle (default), ellipse, square, or circle. If square or circle is used, the height of the control is ignored (height equals width).
    • color: Color? info - The background color of the button.
    • textColor: Color? info - The text color of the button.
    • landscape: Boolean? info - If set to true, rotates the button 90 degrees around its top left corner.
    • fontSize: FontSize? info - The size of the font to use for text (default 1.0).

Returns: String info - id of the created control

PhoneIoT.addImageDisplay(device: Device, x: Position, y: Position, width: Size, height: Size, options: Object?)

Adds an image display wit hthe given position and size. If not specified, an image display is readonly, meaning that the user cannot modify its content. If (explicitly) not set to readonly, then the user can click on the image display to change the image to a new picture from the camera.

Arguments:

  • device: Device info - id of the device

  • x: Position info - X position of the top left corner of the image display (percentage).

  • y: Position info - Y position of the top left corner of the image display (percentage).

  • width: Size info - Width of the image display (percentage).

  • height: Size info - Height of the image display (percentage).

  • options: Object? info - Additional options

    • id: String? info - The id to use for the control. If not specified, a new one will be automatically generated.
    • event: String? info - The name of a message type to be sent each time the user updates the content (only possible if readonly = false). You must call PhoneIoT.listenToGUI() to actually receive these messages. If not specified, no event is sent. Message fields: id.
    • readonly: Boolean? info - Specifies if the user is allowed to change the content (defaults to true). Regardless of this setting, you can still modify the image programmatically via PhoneIoT.setImage(). Defaults to true.
    • landscape: Boolean? info - If set to true, rotates the image display 90 degrees around its top left corner.
    • fit: Fit? info - The technique used to fit the image into the display, in case the image and the display have different aspect ratios. This can be fit (default), zoom, or stretch.

Returns: String info - id of the created control

PhoneIoT.addJoystick(device: Device, x: Position, y: Position, width: Size, options: Object?)

Adds a joystick control to the canvas at the given position and size. No height parameter is given because joysticks are always circular (similar to passing style = circle to PhoneIoT.addButton()).

The position of the joystick is given by a vector [x, y], which is normalized to a length of 1. If you would prefer to not have this normalization and want rectangular coordinates instead of circular, consider using PhoneIoT.addTouchpad() instead.

Arguments:

  • device: Device info - id of the device

  • x: Position info - X position of the top left corner of the joystick (percentage).

  • y: Position info - Y position of the top left corner of the joystick (percentage).

  • width: Size info - Width of the joystick (percentage).

  • options: Object? info - Additional options

    • id: String? info - The id to use for the control. If not specified, a new one will be automatically generated.
    • event: String? info - The name of a message type to be sent each time the user moves the joystick. The messages also include a tag field which functions identically to the one in PhoneIoT.addTouchpad(). You must call PhoneIoT.listenToGUI() to actually receive these messages. If not specified, no event is sent. Message fields: id, x, y, tag.
    • color: Color? info - The color of the joystick.
    • landscape: Boolean? info - If set to true, the x and y values of the joystick are altered so that it acts correctly when in landscape mode. Unlike other controls, this option does not affect where the control is displayed on the screen (no rotation).

Returns: String info - id of the created control

PhoneIoT.addLabel(device: Device, x: Position, y: Position, text: String?, options: Object?)

Adds a label control to the canvas at the given position. If text is not specified, it default to empty, which can be used to hide the label when nothing needs to be displayed. The text can be modified later via PhoneIoT.setText().

Labels do not have a size, so they also don’t do text wrapping. Because of this, you should keep label text relatively short. If you need a large amount of text written, consider using PhoneIoT.addTextField() with readonly = true.

Arguments:

  • device: Device info - id of the device

  • x: Position info - X position of the top left corner of the label (percentage).

  • y: Position info - Y position of the top left corner of the label (percentage).

  • text: String? info - The text to display on the label (defaults to empty)

  • options: Object? info - Additional options

    • id: String? info - The id to use for the control. If not specified, a new one will be automatically generated.
    • textColor: Color? info - The text color of the label.
    • align: Align? info - The text alignment to use. If set to left, the text starts at the label position. If set to right, the text ends at the label position. If set to center, the text is centered on the label position.
    • fontSize: FontSize? info - The size of the font to use for text (default 1.0).
    • landscape: Boolean? info - If set to true, rotates the label 90 degrees around the label position so the text appears upright when viewed in landscape.

Returns: String info - id of the created control

PhoneIoT.addRadioButton(device: Device, x: Position, y: Position, text: String?, options: Object?)

Adds a radio button to the canvas. Radio buttons are like toggles (checkboxes), except that they are organized into groups and the user can check at most one radion button from any given group. These can be used to accept multiple-choice input from the user.

Arguments:

  • device: Device info - id of the device

  • x: Position info - X position of the top left corner of the radio button (percentage).

  • y: Position info - Y position of the top left corner of the radio button (percentage).

  • text: String? info - The text to display next to the checkbox (defaults to empty)

  • options: Object? info - Additional options

    • group: String? info - The name of the group to associate this radio button with. You do not need this value to access the control later. If not specified, defaults to main.
    • id: String? info - The id to use for the control. If not specified, a new one will be automatically generated.
    • event: String? info - The name of an event to send every time the user clicks the radio button. Note that clicking a radio button always checks it, unlike toggles. You must call PhoneIoT.listenToGUI() to actually receive these messages. If not specified, no event is sent. Message fields: id, state.
    • checked: Boolean? info - Defaults to false. If set to true, the radio button will be initially checked. Note that, while the user cannot check multiple radio buttons, you are free to do so programmatically.
    • color: Color? info - The color of the radio button itself.
    • textColor: Color? info - The text color of the radio button.
    • fontSize: FontSize? info - The size of the font to use for text (default 1.0). Note that this will also scale up the size of the radio button itself (not just the text).
    • landscape: Boolean? info - If set to true, rotates the radio button 90 degrees around its top left corner.
    • readonly: Boolean? info - If set to true, prevents the user from clicking the radio button. However, you will still be free to update the state programmatically. Defaults to false.

Returns: String info - id of the created control

PhoneIoT.addSlider(device: Device, x: Position, y: Position, width: Size, options: Object?)

Adds a slider control to the display. Sliders can be moved around to input or display any value in the range [0, 1]. If you need values outside of this range, you can do a little math to map them to [0, 1] or vice versa.

You can read and write the value of a slider with PhoneIoT.getLevel() and PhoneIoT.setLevel(). Note that if the control is set to readonly, the user cannot change the value, but you can still do so from code.

Arguments:

  • device: Device info - id of the device

  • x: Position info - X position of the top left corner of the slider (percentage).

  • y: Position info - Y position of the top left corner of the slider (percentage).

  • width: Size info - Width (length) of the slider (percentage).

  • options: Object? info - Additional options

    • id: String? info - The id to use for the control. If not specified, a new one will be automatically generated.
    • event: String? info - The name of a message type to be sent each time the user touches, slides, or lets go of the slider. The messages also include a tag field which functions identically to the one in PhoneIoT.addTouchpad(). You must call PhoneIoT.listenToGUI() to actually receive these messages. If not specified, no event is sent. Message fields: id, level, tag.
    • color: Color? info - The color of the slider.
    • value: BoundedNumber<0, 1>? info - The initial value of the slider (default 0.0).
    • style: SliderStyle? info - Controls the appearance of the slider. Allowed values are slider (default) or progress.
    • landscape: Boolean? info - true to rotate the control 90 degrees into landscape mode.
    • readonly: Boolean? info - true to disable the user from controlling the slider. This is especially usefull for progress bars.

Returns: String info - id of the created control

PhoneIoT.addTextField(device: Device, x: Position, y: Position, width: Size, height: Size, options: Object?)

Adds a text field to the canvas. These are typically used to display large blocks of text, or to accept input text from the user. If not set to readonly, the user can click on the text field to change its content.

If you have a small amount of text you need to show and would otherwise make this control readonly, consider using PhoneIoT.addLabel() instead.

Arguments:

  • device: Device info - id of the device

  • x: Position info - X position of the top left corner of the text field (percentage).

  • y: Position info - Y position of the top left corner of the text field (percentage).

  • width: Size info - Width of the text field (percentage).

  • height: Size info - Height of the text field (percentage).

  • options: Object? info - Additional options

    • id: String? info - The id to use for the control. If not specified, a new one will be automatically generated.
    • event: String? info - The name of an event to send every time the user changes the text content (only possible if readonly = false). Note that this event is only sent once the user clicks accept on the new content (you do not get an event for every key press). You must call PhoneIoT.listenToGUI() to actually receive these messages. If not specified, no event is sent. Message fields: id, text.
    • text: String? info - This can be used to set the initial text of the text field once created. Defaults to empty if not specified.
    • color: Color? info - The color of the text field border.
    • textColor: Color? info - The text color of the text field.
    • readonly: Boolean? info - If set to true, the user will not be able to edit the content. However, you will still be free to do so programmatically. Defaults to false.
    • fontSize: FontSize? info - The size of the font to use for text (default 1.0).
    • align: Align? info - The text alignment to use. This can be left (default), right, or center.
    • landscape: Boolean? info - If set to true, rotates the text field 90 degrees around its top left corner.

Returns: String info - id of the created control

PhoneIoT.addToggle(device: Device, x: Position, y: Position, text: String?, options: Object?)

Adds a toggle control to the canvas at the given location. The text parameter can be used to set the initial text shown for the toggle (defaults to empty), but this can be changed later with PhoneIoT.setText().

Arguments:

  • device: Device info - id of the device

  • x: Position info - X position of the top left corner of the toggle (percentage).

  • y: Position info - Y position of the top left corner of the toggle (percentage).

  • text: String? info - The text to display next to the toggle (defaults to empty)

  • options: Object? info - Additional options

    • style: ToggleStyle? info - The visual style of the toggle control. This can be switch (default) for a mobile-style toggle, or checkbox for a desktop-style toggle.
    • id: String? info - The id to use for the control. If not specified, a new one will be automatically generated.
    • event: String? info - The name of a message to be sent every time the checkbox is toggled by the user. You must call PhoneIoT.listenToGUI() to actually receive these messages. Message fields: id, state.
    • checked: Boolean? info - Defaults to false. If set to true, the toggle will be initially checked.
    • color: Color? info - The color of the toggle itself.
    • textColor: Color? info - The text color of the toggle.
    • fontSize: FontSize? info - The size of the font to use for text (default 1.0). Note that this will also scale up the size of the toggle itself (not just the text).
    • landscape: Boolean? info - If set to true, rotates the toggle 90 degrees around its top left corner.
    • readonly: Boolean? info - If set to true, prevents the user from clicking the toggle. However, you will still be free to update the state programmatically. Defaults to false.

Returns: String info - id of the created control

PhoneIoT.addTouchpad(device: Device, x: Position, y: Position, width: Size, height: Size, options: Object?)

Adds a touchpad control to the canvas at the given position and size. This control is similar to the joystick control, except that it is rectangular, the vector is not normalized to a distance of 1, the “stick” does not move back to (0, 0) upon letting go, and there is an additional “tag” value denoting if each event was a touch down, move, or up.

Although the vector value is not normalized to a length of 1, each component (x and y individually) is in [-1, 1].

Arguments:

  • device: Device info - id of the device

  • x: Position info - X position of the top left corner of the touchpad (percentage).

  • y: Position info - Y position of the top left corner of the touchpad (percentage).

  • width: Size info - Width of the touchpad (percentage).

  • height: Size info - Height of the touchpad (percentage).

  • options: Object? info - Additional options

    • id: String? info - The id to use for the control. If not specified, a new one will be automatically generated.
    • event: String? info - The name of a message type to be sent each time the user touches, slides, or lets go of the touchpad. A message field called tag is included to differentiate the different types of interactions; it is one of down (touch started), up (touch ended), or move (during continued/held touch). You must call PhoneIoT.listenToGUI() to actually receive these messages. If not specified, no event is sent. Message fields: id, x, y, tag.
    • color: Color? info - The color of the touchpad.
    • style: TouchpadStyle? info - Controls the appearance of the touchpad. These are the same as for PhoneIoT.addButton() except that only rectangle and square are allowed.
    • landscape: Boolean? info - true to rotate the control 90 degrees into landscape mode.

Returns: String info - id of the created control

PhoneIoT.clearControls(device: Device)

Removes all controls from the device’s canvas. If you would instead like to remove a specific control, see PhoneIoT.removeControl().

Arguments:

  • device: Device info - id of the device
PhoneIoT.getImage(device: Device, id: String)

Gets the displayed image of an image-like control with the given ID. This can be used on any control that displays images, which is currently only image displays.

This can be used to retrieve images from the mobile device’s camera, by having the user store an image in an image display that has readonly = false. See the readonly optional parameter of PhoneIoT.addImageDisplay().

Arguments:

  • device: Device info - id of the device
  • id: String info - id of the image display

Returns: Image - the displayed image

PhoneIoT.getLevel(device: Device, id: String)

Get the current value (a single number) of a value-like control. Currently, the only supported control is a slider, which returns a value in [0, 1].

Instead of calling this in a loop, it is likely better to use the event optional parameter of PhoneIoT.addSlider().

If you want to get the cursor position of a joystick or touchpad, use PhoneIoT.getPosition() instead.

Arguments:

  • device: Device info - id of the device
  • id: String info - id of the control to read

Returns: BoundedNumber<0, 1> info - current value

PhoneIoT.getPosition(device: Device, id: String)

Gets the current x and y values for the current position of a positional control. This does not give the location of the control on the screen. Positional controls are controls whose primary interaction is through position. For instance, this is used for both joystick and touchpad controls.

For a joystick, this always returns a vector normalized to a length of 1.0. If the user is not touching the joystick, it will automatically go back to the center, [0, 0].

For a touchpad, this will either give you the current location of the touch (a list of [x, y]) or an error if the user is not touching the screen.

If you want to get the value of a slider, use PhoneIoT.getLevel() instead.

Instead of calling this in a loop, it is likely better to use the event optional parameter of PhoneIoT.addJoystick() or PhoneIoT.addTouchpad().

Arguments:

  • device: Device info - id of the device
  • id: String info - id of the control to read

Returns: List<BoundedNumber<-1, 1>> info - a list of [x, y] for the current position, or a string explaining that there is no current position

PhoneIoT.getText(device: Device, id: String)

Gets the current text content of the text-like control with the given ID. This can be used on any control that has text, such as a button, label, or text field.

Arguments:

  • device: Device info - id of the device
  • id: String info - id of the control to read

Returns: String info - currently displayed text

PhoneIoT.getToggleState(device: Device, id: String)

Gets the toggle state of a toggleable control. This can be used on any toggleable control, such as toggles and radio buttons.

Arguments:

  • device: Device info - id of the device
  • id: String info - id of the control to read

Returns: Boolean info - true for checked, otherwise false

PhoneIoT.isPressed(device: Device, id: String)

Checks if the pressable control with the given ID is currently pressed. This can be used on any pressable control, which currently includes buttons, joysticks, and touchpads.

By calling this RPC in a loop, you could perform some action every second while a button is held down. If you would instead like to receive click events, see the event optional parameter of PhoneIoT.addButton().

Arguments:

  • device: Device info - id of the device
  • id: String info - id of the control to read

Returns: Boolean info - true for pressed, otherwise false

PhoneIoT.removeControl(device: Device, id: String)

Removes a control with the given ID if it exists. If the control does not exist, does nothing (but still counts as success). If you would instead like to remove all controls, see PhoneIoT.clearControls().

Arguments:

  • device: Device info - id of the device
  • id: String info - id of the control to remove
PhoneIoT.setImage(device: Device, id: String, img: Image)

Sets the displayed image of an image-like control with the given ID. This can be used on any control that displays images, which is currently only image displays.

Arguments:

  • device: Device info - id of the device
  • id: String info - the id of the control to modify
  • img: Image - the new image to display
PhoneIoT.setLevel(device: Device, id: String, value: BoundedNumber<0, 1>)

Set the current value (a single number) of a value-like control. Currently, the only supported control is a slider, which sets the displayed value.

Note that you can use this RPC even if the control is set to readonly mode (it’s only readonly to the user).

Arguments:

  • device: Device info - id of the device
  • id: String info - id of the control to read
  • value: BoundedNumber<0, 1> info - new value to set

Returns: Number info - current value

PhoneIoT.setText(device: Device, id: String, text: String?)

Sets the text content of the text-like control with the given ID. This can be used on any control that has text, such as a button, label, or text field.

Arguments:

  • device: Device info - id of the device
  • id: String info - id of the control to modify
  • text: String? info - The new text to display (defaults to empty)
PhoneIoT.setToggleState(device: Device, id: String, state: Boolean)

Sets the toggle state of a toggleable control with the given ID. This can be used on any toggleable control, such as toggles and radio buttons. If state is true, the toggleable becomes checked, otherwise it is unchecked.

If used on a radio button, it sets the state independent of the control’s group. That is, although the user can’t select multiple radio buttons in the same group, you can do so programmatically through this RPC.

Arguments:

  • device: Device info - id of the device
  • id: String info - id of the control to modify
  • state: Boolean info - new value for the toggle state