Display

This section covers all of the RPCs that relate to manipulation of the customizable interactive display on the mobile device. The device display is a large rectangular region called the canvas.

Locations on the canvas are defined by x and y coordinates that range from [0, 100]. These coordinates are relative to the size of the display. Thus, a button at position [25, 25] and size [50, 50] will take up a fourth of the screen (half the width and half the height), and will be centered in the canvas.

Many controls take optional parameters, which are specified as a list of pairs (lists of size 2), where each pair is the name of an option and the desired value. For instance, most controls take one or more optional parameters to control the color of the display. You can obtain color codes from PhoneIoT.getColor().

RPCS

PhoneIoT.addButton(device: Device, x: Position, y: Position, width: Size, height: Size, text: String?, options: Object?)

Adds a button to the display with the given position and size. If not specified, the default text for a button is empty, which can be used to just make a colored, unlabeled button. The text can be modified later via PhoneIoT.setText().

Adding a widget/control to the device will automatically call PhoneIoT.listenToGUI(), which is needed to receive events from user interaction. If you have a scenario where one project sets up the GUI but another project also needs to listen for GUI events, the other project can directly call PhoneIoT.listenToGUI() to receive the events without needing to actually add a widget.

Arguments:

  • device: Device (Device) - id of the device

  • x: Position (Position) - X position of the top left corner of the button (percentage)

  • y: Position (Position) - Y position of the top left corner of the button (percentage)

  • width: Size (Size) - Width of the button (percentage)

  • height: Size (Size) - Height of the button (percentage)

  • text: String? (String) - text to display on the button (default empty)

  • options: Object? (Object) - Additional options

    • id: String? (String) - The id to use for the control. If not specified, a new one will be automatically generated.

    • event: String? (String) - The name of a message type to be sent each time the button is pressed. You must call PhoneIoT.listenToGUI() to actually receive these messages. If not specified, no event is sent. Message fields: device, id.

    • style: ButtonStyle? (ButtonStyle) - The display style of the button on the screen. This can be rectangle (default), ellipse, square, or circle. If square or circle is used, the height of the control is ignored (height equals width).

    • color: Color? (Color) - The background color of the button.

    • textColor: Color? (Color) - The text color of the button.

    • landscape: Boolean? (Boolean) - If set to true, rotates the button 90 degrees around its top left corner.

    • fontSize: FontSize? (FontSize) - The size of the font to use for text (default 1.0).

Returns: String (String) - id of the created control

PhoneIoT.addImageDisplay(device: Device, x: Position, y: Position, width: Size, height: Size, options: Object?)

Adds an image display with the given position and size. If not specified, an image display is by default readonly = true, meaning that the user cannot modify its content. If (explicitly) set to readonly = false, then the user can click on the image display to change the image to a new picture from the camera.

Adding a widget/control to the device will automatically call PhoneIoT.listenToGUI(), which is needed to receive events from user interaction. If you have a scenario where one project sets up the GUI but another project also needs to listen for GUI events, the other project can directly call PhoneIoT.listenToGUI() to receive the events without needing to actually add a widget.

Arguments:

  • device: Device (Device) - id of the device

  • x: Position (Position) - X position of the top left corner of the image display (percentage).

  • y: Position (Position) - Y position of the top left corner of the image display (percentage).

  • width: Size (Size) - Width of the image display (percentage).

  • height: Size (Size) - Height of the image display (percentage).

  • options: Object? (Object) - Additional options

    • id: String? (String) - The id to use for the control. If not specified, a new one will be automatically generated.

    • event: String? (String) - The name of a message type to be sent each time the user updates the content (only possible if readonly = false). You must call PhoneIoT.listenToGUI() to actually receive these messages. If not specified, no event is sent. Message fields: device, id.

    • readonly: Boolean? (Boolean) - If set to true (default), the user will not be able to edit the content; however, you will still be able to do so programmatically via PhoneIoT.setImage(). Defaults to true.

    • landscape: Boolean? (Boolean) - If set to true, rotates the image display 90 degrees around its top left corner.

    • fit: Fit? (Fit) - The technique used to fit the image into the display, in case the image and the display have different aspect ratios. This can be fit (default), zoom, or stretch.

Returns: String (String) - id of the created control

PhoneIoT.addJoystick(device: Device, x: Position, y: Position, width: Size, options: Object?)

Adds a joystick control to the canvas at the given position and size. No height parameter is given because joysticks are always circular (similar to passing style = circle to PhoneIoT.addButton()).

The position of the joystick is given by a vector [x, y], which is normalized to a length of 1. If you would prefer to not have this normalization and want rectangular coordinates instead of circular, consider using PhoneIoT.addTouchpad() instead.

Adding a widget/control to the device will automatically call PhoneIoT.listenToGUI(), which is needed to receive events from user interaction. If you have a scenario where one project sets up the GUI but another project also needs to listen for GUI events, the other project can directly call PhoneIoT.listenToGUI() to receive the events without needing to actually add a widget.

Arguments:

  • device: Device (Device) - id of the device

  • x: Position (Position) - X position of the top left corner of the joystick (percentage).

  • y: Position (Position) - Y position of the top left corner of the joystick (percentage).

  • width: Size (Size) - Width of the joystick (percentage).

  • options: Object? (Object) - Additional options

    • id: String? (String) - The id to use for the control. If not specified, a new one will be automatically generated.

    • event: String? (String) - The name of a message type to be sent each time the user moves the joystick. The messages also include a tag field which functions identically to the one in PhoneIoT.addTouchpad(). You must call PhoneIoT.listenToGUI() to actually receive these messages. If not specified, no event is sent. Message fields: device, id, x, y, tag.

    • color: Color? (Color) - The color of the joystick.

    • landscape: Boolean? (Boolean) - If set to true, the x and y values of the joystick are altered so that it acts correctly when in landscape mode. Unlike other controls, this option does not affect where the control is displayed on the screen (no rotation).

Returns: String (String) - id of the created control

PhoneIoT.addLabel(device: Device, x: Position, y: Position, text: String?, options: Object?)

Adds a label control to the canvas at the given position. If text is not specified, it default to empty, which can be used to hide the label when nothing needs to be displayed. The text can be modified later via PhoneIoT.setText().

Labels do not have a size, so they also don’t do text wrapping. Because of this, you should keep label text relatively short. If you need a large amount of text written, consider using PhoneIoT.addTextField() with readonly = true.

Adding a widget/control to the device will automatically call PhoneIoT.listenToGUI(), which is needed to receive events from user interaction. If you have a scenario where one project sets up the GUI but another project also needs to listen for GUI events, the other project can directly call PhoneIoT.listenToGUI() to receive the events without needing to actually add a widget.

Arguments:

  • device: Device (Device) - id of the device

  • x: Position (Position) - X position of the top left corner of the label (percentage).

  • y: Position (Position) - Y position of the top left corner of the label (percentage).

  • text: String? (String) - The text to display on the label (defaults to empty)

  • options: Object? (Object) - Additional options

    • id: String? (String) - The id to use for the control. If not specified, a new one will be automatically generated.

    • textColor: Color? (Color) - The text color of the label.

    • align: Align? (Align) - The text alignment to use. If set to left, the text starts at the label position. If set to right, the text ends at the label position. If set to center, the text is centered on the label position.

    • fontSize: FontSize? (FontSize) - The size of the font to use for text (default 1.0).

    • landscape: Boolean? (Boolean) - If set to true, rotates the label 90 degrees around the label position so the text appears upright when viewed in landscape.

Returns: String (String) - id of the created control

PhoneIoT.addRadioButton(device: Device, x: Position, y: Position, text: String?, options: Object?)

Adds a radio button to the canvas. Radio buttons are like toggles (checkboxes), except that they are organized into groups and the user can check at most one radio button from any given group. These can be used to accept multiple-choice input from the user.

Adding a widget/control to the device will automatically call PhoneIoT.listenToGUI(), which is needed to receive events from user interaction. If you have a scenario where one project sets up the GUI but another project also needs to listen for GUI events, the other project can directly call PhoneIoT.listenToGUI() to receive the events without needing to actually add a widget.

Arguments:

  • device: Device (Device) - id of the device

  • x: Position (Position) - X position of the top left corner of the radio button (percentage).

  • y: Position (Position) - Y position of the top left corner of the radio button (percentage).

  • text: String? (String) - The text to display next to the checkbox (defaults to empty)

  • options: Object? (Object) - Additional options

    • group: String? (String) - The name of the group to associate this radio button with. You do not need this value to access the control later. If not specified, defaults to main.

    • id: String? (String) - The id to use for the control. If not specified, a new one will be automatically generated.

    • event: String? (String) - The name of an event to send every time the user clicks the radio button. Note that clicking a radio button always checks it, unlike toggles. You must call PhoneIoT.listenToGUI() to actually receive these messages. If not specified, no event is sent. Message fields: device, id.

    • checked: Boolean? (Boolean) - Defaults to false. If set to true, the radio button will be initially checked. Note that, while the user cannot check multiple radio buttons, you are free to do so programmatically.

    • color: Color? (Color) - The color of the radio button itself.

    • textColor: Color? (Color) - The text color of the radio button.

    • fontSize: FontSize? (FontSize) - The size of the font to use for text (default 1.0). Note that this will also scale up the size of the radio button itself (not just the text).

    • landscape: Boolean? (Boolean) - If set to true, rotates the radio button 90 degrees around its top left corner.

    • readonly: Boolean? (Boolean) - If set to true, the user will not be able to change the state by clicking; however, you will still be free to do so from code. Defaults to false.

Returns: String (String) - id of the created control

PhoneIoT.addSlider(device: Device, x: Position, y: Position, width: Size, options: Object?)

Adds a slider control to the display. Sliders can be moved around to input or display any value in the range [0, 1]. If you need values outside of this range, you can do a little math to map them to [0, 1] or vice versa.

You can read and write the value of a slider with PhoneIoT.getLevel() and PhoneIoT.setLevel(). Note that if the control is set to readonly = true, the user cannot change the value, but you can still do so from code.

Adding a widget/control to the device will automatically call PhoneIoT.listenToGUI(), which is needed to receive events from user interaction. If you have a scenario where one project sets up the GUI but another project also needs to listen for GUI events, the other project can directly call PhoneIoT.listenToGUI() to receive the events without needing to actually add a widget.

Arguments:

  • device: Device (Device) - id of the device

  • x: Position (Position) - X position of the top left corner of the slider (percentage).

  • y: Position (Position) - Y position of the top left corner of the slider (percentage).

  • width: Size (Size) - Width (length) of the slider (percentage).

  • options: Object? (Object) - Additional options

    • id: String? (String) - The id to use for the control. If not specified, a new one will be automatically generated.

    • event: String? (String) - The name of a message type to be sent each time the user touches, slides, or lets go of the slider. The messages also include a tag field which functions identically to the one in PhoneIoT.addTouchpad(). You must call PhoneIoT.listenToGUI() to actually receive these messages. If not specified, no event is sent. Message fields: device, id, level, tag.

    • color: Color? (Color) - The color of the slider.

    • value: BoundedNumber<0, 1>? (BoundedNumber) - The initial value of the slider (default 0.0).

    • style: SliderStyle? (SliderStyle) - Controls the appearance of the slider. Allowed values are slider (default) or progress.

    • landscape: Boolean? (Boolean) - true to rotate the control 90 degrees into landscape mode.

    • readonly: Boolean? (Boolean) - If set to true, the user will not be able to change the value by sliding; however, you are still able to change the value from code. This is especially useful for displaying progress bars such as for a long-running application. Defaults to false.

Returns: String (String) - id of the created control

PhoneIoT.addTextField(device: Device, x: Position, y: Position, width: Size, height: Size, options: Object?)

Adds a text field to the canvas. These are typically used to display large blocks of text, or to accept input text from the user. Unless set to readonly = true, the user can click on the text field to change its content.

If you have a small amount of text you need to show and would otherwise make this control readonly = true, consider using PhoneIoT.addLabel() instead.

Adding a widget/control to the device will automatically call PhoneIoT.listenToGUI(), which is needed to receive events from user interaction. If you have a scenario where one project sets up the GUI but another project also needs to listen for GUI events, the other project can directly call PhoneIoT.listenToGUI() to receive the events without needing to actually add a widget.

Arguments:

  • device: Device (Device) - id of the device

  • x: Position (Position) - X position of the top left corner of the text field (percentage).

  • y: Position (Position) - Y position of the top left corner of the text field (percentage).

  • width: Size (Size) - Width of the text field (percentage).

  • height: Size (Size) - Height of the text field (percentage).

  • options: Object? (Object) - Additional options

    • id: String? (String) - The id to use for the control. If not specified, a new one will be automatically generated.

    • event: String? (String) - The name of an event to send every time the user changes the text content (only possible if readonly = false). Note that this event is only sent once the user clicks accept on the new content (you do not get an event for every key press). You must call PhoneIoT.listenToGUI() to actually receive these messages. If not specified, no event is sent. Message fields: device, id, text.

    • text: String? (String) - This can be used to set the initial text of the text field once created. Defaults to empty if not specified.

    • color: Color? (Color) - The color of the text field border.

    • textColor: Color? (Color) - The text color of the text field.

    • readonly: Boolean? (Boolean) - If set to true, the user will not be able to edit the content; however, you will still be free to do so programmatically. Defaults to false.

    • fontSize: FontSize? (FontSize) - The size of the font to use for text (default 1.0).

    • align: Align? (Align) - The text alignment to use. This can be left (default), right, or center.

    • landscape: Boolean? (Boolean) - If set to true, rotates the text field 90 degrees around its top left corner.

Returns: String (String) - id of the created control

PhoneIoT.addToggle(device: Device, x: Position, y: Position, text: String?, options: Object?)

Adds a toggle control to the canvas at the given location. The text parameter can be used to set the initial text shown for the toggle (defaults to empty), but this can be changed later with PhoneIoT.setText().

Adding a widget/control to the device will automatically call PhoneIoT.listenToGUI(), which is needed to receive events from user interaction. If you have a scenario where one project sets up the GUI but another project also needs to listen for GUI events, the other project can directly call PhoneIoT.listenToGUI() to receive the events without needing to actually add a widget.

Arguments:

  • device: Device (Device) - id of the device

  • x: Position (Position) - X position of the top left corner of the toggle (percentage).

  • y: Position (Position) - Y position of the top left corner of the toggle (percentage).

  • text: String? (String) - The text to display next to the toggle (defaults to empty)

  • options: Object? (Object) - Additional options

    • style: ToggleStyle? (ToggleStyle) - The visual style of the toggle control. This can be switch (default) for a mobile-style toggle, or checkbox for a desktop-style toggle.

    • id: String? (String) - The id to use for the control. If not specified, a new one will be automatically generated.

    • event: String? (String) - The name of a message to be sent every time the checkbox is toggled by the user. You must call PhoneIoT.listenToGUI() to actually receive these messages. Message fields: device, id, state.

    • checked: Boolean? (Boolean) - Defaults to false. If set to true, the toggle will be initially checked.

    • color: Color? (Color) - The color of the toggle itself.

    • textColor: Color? (Color) - The text color of the toggle.

    • fontSize: FontSize? (FontSize) - The size of the font to use for text (default 1.0). Note that this will also scale up the size of the toggle itself (not just the text).

    • landscape: Boolean? (Boolean) - If set to true, rotates the toggle 90 degrees around its top left corner.

    • readonly: Boolean? (Boolean) - If set to true, the user will not be able to change the state by clicking; however, you will still be free to do so from code. Defaults to false.

Returns: String (String) - id of the created control

PhoneIoT.addTouchpad(device: Device, x: Position, y: Position, width: Size, height: Size, options: Object?)

Adds a touchpad control to the canvas at the given position and size. This control is similar to the joystick control, except that it is rectangular, the vector is not normalized to a distance of 1, the “stick” does not move back to (0, 0) upon letting go, and there is an additional “tag” value denoting if each event was a touch down, move, or up.

Although the vector value is not normalized to a length of 1, each component (x and y individually) is in [-1, 1].

Adding a widget/control to the device will automatically call PhoneIoT.listenToGUI(), which is needed to receive events from user interaction. If you have a scenario where one project sets up the GUI but another project also needs to listen for GUI events, the other project can directly call PhoneIoT.listenToGUI() to receive the events without needing to actually add a widget.

Arguments:

  • device: Device (Device) - id of the device

  • x: Position (Position) - X position of the top left corner of the touchpad (percentage).

  • y: Position (Position) - Y position of the top left corner of the touchpad (percentage).

  • width: Size (Size) - Width of the touchpad (percentage).

  • height: Size (Size) - Height of the touchpad (percentage).

  • options: Object? (Object) - Additional options

    • id: String? (String) - The id to use for the control. If not specified, a new one will be automatically generated.

    • event: String? (String) - The name of a message type to be sent each time the user touches, slides, or lets go of the touchpad. A message field called tag is included to differentiate the different types of interactions; it is one of down (touch started), up (touch ended), or move (during continued/held touch). You must call PhoneIoT.listenToGUI() to actually receive these messages. If not specified, no event is sent. Message fields: device, id, x, y, tag.

    • color: Color? (Color) - The color of the touchpad.

    • style: TouchpadStyle? (TouchpadStyle) - Controls the appearance of the touchpad. These are the same as for PhoneIoT.addButton() except that only rectangle and square are allowed.

    • landscape: Boolean? (Boolean) - true to rotate the control 90 degrees into landscape mode.

Returns: String (String) - id of the created control

PhoneIoT.clearControls(device: Device)

Removes all controls from the device’s canvas. If you would instead like to remove a specific control, see PhoneIoT.removeControl().

Arguments:

  • device: Device (Device) - id of the device

PhoneIoT.getImage(device: Device, id: String)

Gets the displayed image of an image-like control with the given ID. This can be used on any control that displays images, which is currently only image displays.

This can be used to retrieve images from the mobile device’s camera by having the user store an image in an image display that was set to readonly = false. See the readonly optional parameter of PhoneIoT.addImageDisplay().

Arguments:

  • device: Device (Device) - id of the device

  • id: String (String) - id of the image display

Returns: Image (Image) - the displayed image

PhoneIoT.getLevel(device: Device, id: String)

Get the current value (a single number) of a value-like control. Currently, the only supported control is a slider, which returns a value in [0, 1].

Instead of calling this in a loop, it is likely better to use the event optional parameter of PhoneIoT.addSlider().

If you want to get the cursor position of a joystick or touchpad, use PhoneIoT.getPosition() instead.

Arguments:

  • device: Device (Device) - id of the device

  • id: String (String) - id of the control to read

Returns: BoundedNumber<0, 1> (BoundedNumber) - current value

PhoneIoT.getPosition(device: Device, id: String)

Gets the current x and y values for the current position of a positional control. This does not give the location of the control on the screen. Positional controls are controls whose primary interaction is through position. For instance, this is used for both joystick and touchpad controls.

For a joystick, this always returns a vector normalized to a length of 1.0. If the user is not touching the joystick, it will automatically go back to the center, [0, 0].

For a touchpad, this will either give you the current location of the touch (a list of [x, y]) or an error if the user is not touching the screen.

If you want to get the value of a slider, use PhoneIoT.getLevel() instead.

Instead of calling this in a loop, it is likely better to use the event optional parameter of PhoneIoT.addJoystick() or PhoneIoT.addTouchpad().

Arguments:

  • device: Device (Device) - id of the device

  • id: String (String) - id of the control to read

Returns: List<BoundedNumber<-1, 1>> (List | BoundedNumber) - a list of [x, y] for the current position, or a string explaining that there is no current position

PhoneIoT.getText(device: Device, id: String)

Gets the current text content of the text-like control with the given ID. This can be used on any control that has text, such as a button, label, or text field.

Arguments:

  • device: Device (Device) - id of the device

  • id: String (String) - id of the control to read

Returns: String (String) - currently displayed text

PhoneIoT.getToggleState(device: Device, id: String)

Gets the toggle state of a toggleable control. This can be used on any toggleable control, such as toggles and radio buttons.

Arguments:

  • device: Device (Device) - id of the device

  • id: String (String) - id of the control to read

Returns: Boolean (Boolean) - true for checked, otherwise false

PhoneIoT.isPressed(device: Device, id: String)

Checks if the pressable control with the given ID is currently pressed. This can be used on any pressable control, which currently includes buttons, joysticks, and touchpads.

By calling this RPC in a loop, you could perform some action every second while a button is held down. If you would instead like to receive click events, see the event optional parameter of PhoneIoT.addButton().

Arguments:

  • device: Device (Device) - id of the device

  • id: String (String) - id of the control to read

Returns: Boolean (Boolean) - true for pressed, otherwise false

PhoneIoT.removeControl(device: Device, id: String)

Removes a control with the given ID if it exists. If the control does not exist, does nothing (but still counts as success). If you would instead like to remove all controls, see PhoneIoT.clearControls().

Arguments:

  • device: Device (Device) - id of the device

  • id: String (String) - id of the control to remove

PhoneIoT.setImage(device: Device, id: String, img: Image)

Sets the displayed image of an image-like control with the given ID. This can be used on any control that displays images, which is currently only image displays.

Arguments:

  • device: Device (Device) - id of the device

  • id: String (String) - the id of the control to modify

  • img: Image (Image) - the new image to display

PhoneIoT.setLevel(device: Device, id: String, value: BoundedNumber<0, 1>)

Set the current value (a single number) of a value-like control. Currently, the only supported control is a slider, which sets the displayed value.

Note that you can use this RPC even if the control was set to readonly = true (readonly is only a restriction for the user).

Arguments:

  • device: Device (Device) - id of the device

  • id: String (String) - id of the control to read

  • value: BoundedNumber<0, 1> (BoundedNumber) - new value to set

Returns: Number (Number) - current value

PhoneIoT.setText(device: Device, id: String, text: String?)

Sets the text content of the text-like control with the given ID. This can be used on any control that has text, such as a button, label, or text field.

Arguments:

  • device: Device (Device) - id of the device

  • id: String (String) - id of the control to modify

  • text: String? (String) - The new text to display (defaults to empty)

PhoneIoT.setToggleState(device: Device, id: String, state: Boolean)

Sets the toggle state of a toggleable control with the given ID. This can be used on any toggleable control, such as toggles and radio buttons. If state is true, the toggleable becomes checked, otherwise it is unchecked.

If used on a radio button, it sets the state independent of the control’s group. That is, although the user can’t select multiple radio buttons in the same group, you can do so programmatically through this RPC.

Arguments:

  • device: Device (Device) - id of the device

  • id: String (String) - id of the control to modify

  • state: Boolean (Boolean) - new value for the toggle state