This is version 0.1.0 of ShareAR. Find another version
The ShareAR toolkit facilitates secure and private content sharing between users of a multi-user augmented reality (AR) application. Users do unexpected things: they might try to see virtual content they shouldn’t have access to, or they might try to spam another user with obnoxious virtual content. Does this sound like a bad user experience? Might this be possible in your application? Then read on!
Let’s say you’re writing an app for collaborative architecture design. You want to make a 3D model only visible to the project’s contributors, and you want to prevent some user from littering the working space with offensive content that other users can see but not edit or remove. Instead of coding up a permission matrix or an offer/accept sharing flow yourself, simply connect up your app code and virtual models to the ShareAR system and call the ShareAR API. To share an object myModel with a collaborator identified by the ID 42, write:
PermissionSystem.Instance.TryChangePermission(42, myModel, true, PermissionSystem.Permission.EDIT);
ShareAR uses an offer/accept sharing flow as its default behavior, but bypassing that flow for a trusted collaborator is as easy as:
PermissionSystem.Instance.SetTrustContentSource(42, true);
ShareAR makes multi-user AR security and privacy easier. Read Section 2 to see how to connect it up to your app, Section 3 to understand the big ideas it’s built on, and Section 4 for the full API reference docs. If you find a bug, we’d love to hear about it -- see Section 5 for known issues and Section 6 for ways to contribute to the project.
This toolkit assumes a Unity-based AR development workflow. Scripts are written in C# and call Unity APIs directly. ShareAR was built for use in apps written for the Microsoft HoloLens, but it does not directly depend on HoloLens APIs. The setup instructions below differ based on what method is used to establish a network connection between devices.
ShareAR explicitly supports the HoloLens HoloToolkit Sharing module, which it invokes to send messages between devices. To use ShareAR with the HoloLens Sharing module:
Please refer to Section 5 for known issues and Section 6 for information on contributing.
ShareAR has not been tested on other AR platforms or other interdevice networking solutions (e.g., Unity Networking), but adaptation to other platforms is designed to be simple. The only known large change to make is to replace PermissionShareInterface.cs with another file of the same name that adheres to the same interface with PermissionSystem.cs. Most methods within PermissionShareInterface.cs consist solely of serializing and deserializing collections of method parameters.
Code for compatibility with other platforms or networking solutions is welcome. Please refer to Section 6 for information on contributing to ShareAR.
All objects shared using this toolkit must be created and deleted through designated toolkit methods rather than Unity’s Instantiate and Destroy methods.
Objects are created via the InstantiateShared method. The method takes a Unity prefab to create the shared object from (this prefab must include a SharedObject.cs script component -- see Section 2.1), a user ID to automatically share this object with, the permissions to offer that user ID upon sharing it, and a boolean for whether the object should be shared in a location-decoupled way with that user ID. It returns the created GameObject. For example, creating an object from myPrefab and offering VIEW permissions to a co-located user with an ID of 42 is achieved as follows:
GameObject newObject = PermissionSystem.Instance.InstantiateShared(myPrefab, 42, PermissionSystem.Permission.VIEW, false);
To offer the given permissions to all other participating users instead of only a single one, the constant -1 can be used.
Objects shared in the default location-coupled way (see Section 3.4 for alternatives) are deleted using the DeleteShared method. Deleting an object myObject for one user only, with all other sufficiently permissioned users still able to access the object, is achieved by setting the scope parameter to 0, as follows:
PermissionSystem.Instance.DeleteShared(myObject, 0);
Deleting the object for all users is achieved by setting the scope parameter to 1, as follows:
PermissionSystem.Instance.DeleteShared(myObject, 1);
ShareAR contains permission controls on both a per-object and a per-user basis. That is, whether a certain user has access to a certain object is a function of both the user and the object in question.
For a given object and a given user, the user may have a range of possible permissions on the object. The permissions currently supported by ShareAR, given in the Permission enum, are VIEWGHOST, VIEW, MOVE, COPY, EDIT, RESHARE, and OWN. (Please refer to Section 3.3 for further information on the VIEWGHOST permission.) Some permissions also imply other ones; notating X → Y to mean that a user with permission X on an object must also have permission Y on the object, the permission hierarchy can be diagrammed in the following graph:
Sections 3.2.2 and 3.2.3 discuss how to set defaults for these permissions and how to change these permissions, respectively. Please also refer to Section 3.3 for recommended practices for co-located users, and to Section 3.4 for further information on sharing content with users who may not be co-located.
ShareAR also supports default permission settings that determine the initial permission state when a new object is created or a new user joins.
There are two default axes that define default behavior:
When a new object is created, existing users’ initial permissions on that object are automatically set to be those of the new-object defaults. When a new user joins, their initial permissions on existing objects are automatically set to be those of the new-user defaults.
Both new-user and new-object defaults are initialized with a private access control setting. Applications may override this behavior using the SetDefaultUserPermissions, SetDefaultObjectPermissions, and SetDefaultUserDefaultObjectPermissions methods. However, to minimize unintended information leakage, it is recommended that applications do so cautiously, auto-applying the most restrictive permission settings that are expected to arise realistically in the context of the application.
To change a user’s permission on a specific object, ShareAR provides a TryChangePermission method. Permission revocation calls take effect immediately. Importantly, though, a granted permission does not take effect immediately, but rather must be approved by the user being offered the permission; this allows for the receiving user to manage how their reality is augmented. An example call of TryChangePermission, to revoke view permissions on a GameObject variable myObject to a user with an ID of 42, is shown below:
bool success = PermissionSystem.Instance.TryChangePermission(42, myObject, false, PermissionSystem.Permission.VIEW);
A permission change message from another user is received via either the PermissionChanged event or the NewDiscoverable event. Changes that are auto-processed are relayed to the application via the PermissionChanged event; these include (1) permission revocation messages, (2) permission grants to which preconfigured acceptance rules apply, and (3) permission changes for a third party regarding an object that the current user has permissions on. Offered permission grants that are not yet handled are communicated to the application via the NewDiscoverable event, and it is the responsibility of the application to either accept, ignore, or decline the offer, as detailed below.
Studies of users’ expectations of AR technologies indicate that users tend to draw a strong analogy between shared AR experiences and the shared physical world: notably, users often expect that AR objects in a shared physical space are also shared. However, some content -- such as a private messaging window -- is expected by users to remain visibility-restricted.
ShareAR’s principle of ghosting is a solution to this paradox. Its main idea is to share with other users that an access-restricted object is present in the shared physical space without revealing what sensitive information the object contains. It does so by enabling developer-defined “ghost” objects that serve as location-accurate placeholders for access-restricted objects. Users without permission to view a ghosting-enabled object see instead the ghost object in the same location that the original object is.
Consider, by way of analogy, the scenario of a user interacting with a mobile phone. Even without seeing what is displayed on the phone screen, the visual cue of the phone in the user’s hands (rather than empty air) signals that the user is busy with the device as well as where in physical space the user’s attention is directed, thus allowing other users to scaffold their behavior around that information. Likewise, an AR user who sees another user interacting with a ghost object receives the busy cue and social signal, thereby preserving the user’s intuition about the shared physical world without revealing the sensitive information.
To configure a ghost version of an object in ShareAR, make a prefab for the ghost version and register that prefab with ObjectIDManager.cs just as you would with a non-ghost prefab; then add a reference to the ghost prefab in the original object’s ghostPrefab field in the Unity editor. The ghost prefab should also have a SharedObject.cs script attached.
Once a ghost counterpart to an object prefab pf has been configured, a newly created object myObject with that prefab can be shared in a ghost-only manner by granting all users VIEWGHOST permissions upon myObject’s instantiation as follows:
GameObject myObject = PermissionSystem.Instance.InstantiateShared(pf, AllUsers(), (int)PermissionSystem.Permission.VIEWGHOST, false);
For this policy to apply to all newly joining users, a default permission can be set for the object:
PermissionSystem.Instance.SetDefaultUserPermissions(myObject, (int)PermissionSystem.Permission.VIEWGHOST);
Please refer to Section 3.2.2 for further detail on setting default permissions.
User studies have shown that users often expect that AR objects in a shared physical space obey physical-world intuitions: notably, multiple users all expect to see the same object in the same location. When sharing an object with a co-located user, then, it is ideal that both users see the object in the same location in physical space, and that if one user moves the object then the other users also see the object move. For remotely located users sharing AR content, however, this location-coupling property is impractical; the two users have two separate location-dependent instantiations of the same object, both with the same permissions and the same non-location data.
If two users change their location in the physical world, though, they may become co-located when previously they were not. Under this circumstance, ShareAR ensures that both users see both instantiations of the object such that each instantiation appears in the same physical location to both users. This is shown in the following diagram:
Left: A naive solution for location decoupling; C is not in synch with A and B after moving into the same space. Right: ShareAR’s solution; all users are now in synch.
Creating a location-decoupled object from a prefab pf is done by setting the isRemote parameter in the InstantiateShared method to true, as shown in the following example:
GameObject myObject = PermissionSystem.Instance.InstantiateShared(pf, AllUsers(), PermissionSystem.Permission.VIEW, true);
Sharing myObject in a location-decoupled way with user ID 42 is likewise done by setting the isRemote parameter to true, as follows:
bool success = PermissionSystem.Instance.TryChangePermission(42, myObject, true, PermissionSystem.Permission.EDIT, true);
As with location-coupled object deletion, location-decoupled object deletion is done using the DeleteShared method. It is recommended that this be done in one of two ways. If only the current user’s instantiation of an object myObject is intended to be deleted, leaving all other users’ instantiations intact, then deletion can be done by setting the scope parameter to 1:
PermissionSystem.Instance.DeleteShared(myObject, 1);
If instead myObject is intended to be deleted for all users, destroying all users’ instantiations of the object, then deletion can be done by setting the scope parameter to 2:
PermissionSystem.Instance.DeleteShared(myObject, 2);
This toolkit does not yet support splitting an object into two instantiations of the same object or merging two users’ instantiations into one.
Comfortable with the concepts and just need a quick reference guide? Here are the details:
public event EventHandler<KitchenSinkEventArgs> InstantiateSharedEvent;
Triggered when another user creates an object and shares it with the current user.
public event EventHandler<IDEventArgs> NewGroupedObject;
Triggered when another user shares an object in a location-decoupled way with a third party (thereby creating a new instantiation of the object).
public event EventHandler<IDEventArgs> Deleted;
Triggered when an object is deleted.
public GameObject InstantiateShared(GameObject prefab, int targetUserID, Permission permissions, bool isRemote);
Creates and returns a new object of the given type, such that targetUserID is offered given permissions on the new object. The object will be shared in a location-decoupled way with targetUserID if isRemote=true, else in a location-coupled way. The permission change is not solidified until targetUserID consents to receiving the object. The current user creating the object will be granted OWN permissions on that object. Setting targetUserID to be -1 is a synonym for sharing with all users.
Parameters:
public GameObject InstantiateShared(GameObject prefab, int targetUserID, Permission permissions, bool isRemote, Vector3 position, Quaternion rotation, Vector3 velocity);
Creates and returns a new object of the given type, such that targetUserID is offered given permissions on the new object and the new object has the given physical properties. The object will be shared in a location-decoupled way with targetUserID if isRemote=true, else in a location-coupled way. The permission change is not solidified until targetUserID consents to receiving the object. The current user creating the object will be granted OWN permissions on that object. Setting targetUserID to be -1 is a synonym for sharing with all users.
Parameters:
public GameObject InstantiateShared(GameObject prefab, int targetUserID, Permission permissions, bool isRemote, Vector3 position, Quaternion rotation, Vector3 velocity, string objData);
Creates and returns a new object of the given type, such that targetUserID is offered given permissions on the new object, the new object has the given physical properties, and the new object is configured according to the given string configuration. The object will be shared in a location-decoupled way with targetUserID if isRemote=true, else in a location-coupled way. The permission change is not solidified until targetUserID consents to receiving the object. The current user creating the object will be granted OWN permissions on that object. Setting targetUserID to be -1 is a synonym for sharing with all users.
Parameters:
public GameObject InstantiateShared(GameObject prefab, List<int> targetUserIDs, Permission permissions, bool isRemote, Vector3 position, Quaternion rotation, Vector3 velocity, string objData);
Creates and returns a new object of the given type, such that each user in targetUserIDs is offered given permissions on the new object, the new object has the given physical properties, and the new object is configured according to the given string configuration. The object will be shared in a location-decoupled way with each user in targetUserIDs if isRemote=true, else in a location-coupled way. The permission change is not solidified for a user in targetUserIDs until that user consents to receiving the object. The current user creating the object will be granted OWN permissions on that object. Setting targetUserID to be -1 is a synonym for sharing with all users.
Parameters:
public bool DeleteShared(GameObject obj, int scope);
Deletes given obj according to given scope; returns false iff action fails due to the current user having insufficient permissions on the object. A scope of 0 deletes the object only for the current user; a scope of 1 deletes this co-located instantiation of the object for all users; a scope of 2 deletes the object for all users, co-located or not.
Parameters:
public enum Permission { VIEWGHOST, VIEW, COPY, EDIT, RESHARE, OWN }
Enum representing possible user permissions on an object. Most are self-explanatory. RESHARE means permission to change the sharing settings on the object.
public event EventHandler
Triggered when another user’s permissions on an object change.
public bool CanSetPermission(long affectedUserID, GameObject obj);
Returns true iff the current user is allowed to set affectedUserID’s permissions on obj. Fails if current user does not have RESHARE permissions on obj or if affectedUserID is the current user’s ID.
Parameters:
public bool TryChangePermission(long userID, GameObject obj, bool isGranting, Permission permission, bool isRemote = false);
Attempts to change userID’s permission on obj by applying a permission differential to the user’s current permissions on obj. Permissions can be granted or revoked, and can be applied in a location-coupled or location-decoupled way. Returns false iff current user does not have authority to make the change. Permission change is applied immediately if a revocation; otherwise, is sent to userID for consent and is not guaranteed to be enacted.
Parameters:
public Permission GetPermissions(GameObject obj, long userID);
Returns userID’s permissions on obj.
Parameters:
public bool CanViewGhostOnly(GameObject obj, long userID);
Returns whether userID has VIEWGHOST permissions on obj but no higher permissions on obj.
Parameters:
public bool CanViewGhostOnly(Permission permissions);
Returns whether the given permissions contain VIEWGHOST permissions but no higher permissions.
Parameters:
public Permission AllPermissions();
Returns enum value combining all available permissions. (This can be thought of as a bitstring of all 1s whose length is the number of permissions in the Permission enum.)
Parameters: none.
public void SetDefaultUserPermissions(GameObject obj, Permission permissions);
Sets what permissions a newly-joined user should have on obj by default. CAUTION: changes to this setting are not automatically synchronized among users.
Parameters:
public void SetDefaultObjectPermissions(long userID, Permission permissions);
Sets what permissions userID should have on a newly-created object by default. CAUTION: changes to this setting are not automatically synchronized among users.
Parameters:
public void SetDefaultUserDefaultObjectPermissions(Permission permissions);
Sets what permissions any user should have on any object by default. CAUTION: it is strongly recommended to set this to the most restrictive permissions that are expected to arise in the normal course of the app’s operation. CAUTION: changes to this setting are not automatically synchronized among users.
Parameters:
public event EventHandler<IDGenericEventArgs> UpdateData;
Triggered when the data is updated for an object that the current user has nonzero permissions on.
public event EventHandler<PhysicsEventArgs> UpdateLocation;
Triggered when the location is updated for an object that the current user has nonzero permissions on.
public bool CanUpdateData(GameObject obj);
Returns true iff current user has sufficient permissions on obj to change data of obj. Criterion: user must have EDIT permissions on obj.
Parameters:
public bool CanUpdateLocation(GameObject obj);
Returns true iff current user has sufficient permissions on obj to change location of obj. Criterion: user must have EDIT permissions on obj.
Parameters:
public bool SendUpdateLocation(GameObject obj);
Extracts location of given object and sends location to all users with nonzero permissions on the object. Returns true if change was successfully made and communicated; false otherwise. Fails if CanUpdateLocation would, or if obj does not have sharing script attached.
Parameters:
public bool SendUpdateData(GameObject obj);
Returns true if change was successfully made and communicated; false otherwise. Fails if CanUpdateData would, or if obj does not have sharing script attached.
Parameters:
public class Discoverable;
Summarizes data about an object that has been offered to the current user.
Fields:
public event EventHandler<IDEventArgs> Accepted;
Triggered when a user with whom we have offered shared content accepts the shared content.
public event EventHandler<IDEventArgs> Declined;
Triggered when a user with whom we have offered shared content declines the shared content.
public event EventHandler<DiscoverableEventArgs> NewDiscoverable;
Triggered when another user offers shared content to the current user.
public void Discover(Discoverable disc);
Accepts the offered Discoverable object. Offered permissions on that object are enacted, and the permission change is communicated to all relevant users.
Parameters:
public void Decline(Discoverable disc);
Rejects the offered Discoverable object and communicates rejection to user who offered the object. This method is intended for apps where the user offering the content may see the offer as pending until a response is received from the user to whom the content is offered. For broadcast-style sharing, see the Ignore method.
Parameters:
public void Ignore(Discoverable disc);
Rejects the offered Discoverable object without communicating rejection to user who offered the object. This method is intended for apps where the user sharing the content does so in a broadcast style. Calling this method is recommended when an object is rejected since it releases system resources associated with the pending offer.
Parameters:
public bool Trusted(long fromUserID, GameObject prefab);
Returns whether the current user auto-trusts objects of the given object type from the given user.
Parameters:
public void SetTrustContentSource(long userID, bool doTrust);
Sets whether we automatically trust objects from the given user. Does not override exceptions to the rule as set by SetTrustObjectTypeFromContentSource.
Parameters:
public void SetTrustObjectType(GameObject prefab, bool doTrust);
Sets whether we automatically trust objects of the given type. Does not override exceptions to the rule as set by SetTrustObjectTypeFromContentSource.
Parameters:
public void SetDefaultTrust(bool doTrust);
Sets whether we automatically trust objects shared with the current user. Does not override exceptions to the rule as set by SetTrustContentSource, SetTrustObjectType, or SetTrustObjectTypeFromContentSource. CAUTION: it is strongly recommended to only set doTrust=true in circumstances where object sharing is guaranteed to be limited in volume and semantically devoid of sensitive information.
Parameters:
public void SetTrustObjectTypeFromContentSource(GameObject prefab, long userID, bool doTrust);
Sets whether we automatically trust objects of the given type from the given user.
Parameters:
public event EventHandler<GenericEventArgs> GenericMessage;
Triggered when another user sends the current user a generic message.
public void SendGenericMessage(long toUserID, string msgType, string msg);
Sends given string msg, labeled msgType, to given user. The msgType param is provided as a convenience for apps that may be sending generic messages under multiple separate circumstances.
Parameters:
public List
Returns a list of all currently recognized user IDs.
Parameters: none.
public void RejoinNetwork();
Queries other users to gain current information about the state of internal data that is synchronized among users. Intended to be used in cases when current user is out of sync with other users; in particular, after having temporarily lost connection. CAUTION: this method is experimental and should not be relied upon heavily.
Parameters: none.
This section may be updated as more issues arise.
ShareAR is under active development, and contributions are welcome (especially bugfixes). Please contact us at ShareAR@cs.washington.edu if you have changes to the toolkit that you’d like to see incorporated into the next release.
Bug reports, even in the absence of a fix, are also valuable. Please send us specific reproduction instructions, and ideally a minimal working example app showing the bug in action.
We'd also love to hear your feedback on the ShareAR features: what's working and what's harder to manage. We're more than happy to take feature requests as well.