Super-Resolution iPhone Panoramas for Vision Pro

So far the most compelling experiences I’ve had while wearing an Apple Vision Pro have leaned heavily into the immersive abilities of the device. The Immersive Videos are fantastic, but where things really get interesting is when the immersiveness is derived from my own memories. Spatial Video is very cool, but I’m still trying to find my way with how best to record them, but I have absolute clarity about how to record for the other immersive memory experience, Panoramas.

I am a very avid hiker. I am never more at peace than I am on the top of the mountain. The wilderness is my happy place and whenever I leave it, I always long to return. In the visionOS Photos app there is the ability to wrap your panoramic photos around you in a way which gives you a strong sense of being back in the spot the photo was taken.

I love this. For years I’ve been capturing panoramic photos from my favorite scenic overlooks, but the experience of viewing them was always a bit underwhelming. If you look at them on your iPhone/iPad they are nice but completely lack any sense of scale or wonder. The best approach I’ve found so far is to have them printed large scale and then mounted on the wall. My walls are littered with these prints and I’m very fond of walking up to them and standing a few feet away to “take in the view”.

For example this print of Ben Nevis is currently on the back wall of my office.

I capture all of these panoramas on my iPhone (the above image was captured with an iPhone 14 Pro and is printed five feet wide). What became clear very quickly after starting to make large prints of iPhone photos is that resolution is king. The iPhone camera is amazing for its convenience but up until recently the limit of 12MP captures made making compelling large format prints really difficult. But starting with the iPhone 14 Pro we can now capture images up to 48MP, so now we have 4X the pixels to play with.

The iOS Camera app has a default mode for recording panoramas. This is a very clever bit of UI which guides you to sweep your camera across a landscape in a level fashion. The result of this is very good for a quick capture, but unfortunately right now these panoramas are limited to roughly the width of a standard 12MP capture (you shoot panoramas vertically so sensor width becomes the height of the panorama).

Looking at these iPhone panoramas on a Vision Pro is lovely, they have barely enough resolution to give a good sense of being back at the place where the image was captured. However, after the initial WOW! factor has worn off I started to really notice the fuzziness of the presentation. Presenting an image which is around 3900px tall at a conceptual height of about six feet tall just isn’t enough resolution to really feel immersive.

Thankfully because of my aforementioned photo printing experience in addition to having countless standard iOS panoramas, I also have countless super-resolution iPhone panoramas too.

I continue to capture these in the iOS Camera app but instead of using their panorama mode, I just use the regular old camera mode to record a sweep of several individual 48MP photos which I then later stitch together. The results are amazing. That Scotland photo above ended up at 25,326px × 6,609px, or 167 Megapixels. When viewed on a Vision Pro the effect transitions from good to “woah, I’m back in Scotland”.

Last summer I went hiking to the top of Helvellyn in the English Lake District. While I was up there I took two panoramas (well actually I took dozens 🤫, but I’ll show two here). The first was recorded using the standard panorama mode on the iPhone. It ended up being 13,986px × 3,788px (53MP).

Full Resolution (53MP)

I then also recorded the scene as 20 full resolution 48MP photographs. Holding up my phone in vertical mode and slowly turning around, making sure that each photograph slightly overlapped the previous.

I then merged them together in Photoshop using their “Photomerge” feature (though there are countless tools which can do the merging).

The result is an image which is 41,062px × 7,395px (304MP!).

Full Resolution (304MP)

You won’t be able to see the difference in this article view, but if you click through on each of those images you can view them at full resolution and the difference is, quite literally, massive. (If you have a Vision Pro, I’d really recommend tapping through and then saving them to your library and trying it yourself, the difference is really difficult to appreciate until you are in the actual immersion)

This approach was made all the easier in iOS 17 with the addition of the ability to capture photos in the “HEIF Max” format which avoids the added complexity of handling RAW photos. I’m sure that the truly ‘best’ version of this would be to use “ProRAW Max” images, but so far I’ve found my inability to expertly process those to mean the ultimate difference in quality is fairly minimal compared to the default Camera app image processing magic.

Loading up this new super-resolution panorama on my Vision Pro and then swiping between the two (you can swipe phots in the visionOS Photos app by pinching your fingers and flicking them), the difference is meaningful. With this much resolution the panorama feels more like an “Environment” than a photograph. The rocks look sharp and the horizon clear. It really feels like I’m back on this windswept mountain peak.

Here’s a 100% crop comparison of a tiny section in each image. On the left is the super-resolution, on the right the regular iPhone panorama. As you’d expect there is essentially twice the information. In many respects this is the “Retina” screen equivalent.

There are two other great benefits of this approach:

  • Each individual frame is now a full, regular photograph in your photo library and so can be used on its own for sharing or wallpapers. I use the Photo Shuffle lock screen wallpaper system and it regularly shows me images which are middle frames from panoramas.
  • The framing is much more flexible. You can just keep turning around as long as you like, whereas in the iOS Camera panorama mode it has a limited horizontal range. I often will record a full 360º view of the scene. I’ll then end up cropping this down but I can make that decision in the comfort of my office, rather than having to make it on a wind-buffetted, cliff edge.

There are also two big drawbacks:

  • You have to be much more careful about the capture in the moment. You need to overlap the frames and keep your camera level, otherwise the results are going to be lackluster. I’ve several times gotten back from a trip only to discover that a panorama I captured is wonky. You also really want to take the time series very quickly to avoid things shifting between frames. To alleviate this I typically will record a quick one with the Camera app’s mode (for safety) and then switch to grabbing the individual frames. Also, the “spirit level” overlay you can turn on in the Camera app is your friend here to keep you from drifting up or down.
  • You now have work to do in your office after the trip. Photo management is rarely fun, so if you go down this path you’re adding a chore to your life.

A little pro tip I have for anyone who is interested in trying out this approach is to record a ‘marker frame’ before and/or after the section of panorama frames. Otherwise, what will happen is that you’ll end up looking back through your library at a bunch of very similar photos taken from the top of a mountain and struggle to know which images need to be stitched. My approach to this is to take a photograph of my fist right before and after the series. This is logistically very easy to do and then when I’m reviewing my photographs these ugly markers will always jump out to me help me find the frames I’m looking for.

And hey, if you start to pursue this on your next wilderness trips you’ll also end up with photographs you can print in large format and put up on your walls. While I love the immersive feeling of looking at these photographs in visionOS, there is nothing to beat beauty of classic, analog art on your walls.

I would be delighted (and not at all surprised) if this kind of capture came in iOS 18 or the iPhone 16 Pro. It seems highly likely that Apple will do whatever they can to ensure that the panoramas they are collecting will look as awesome as possible in visionOS.

Here are a few other full-resolution images if you’d like to try ‘em out:

From Stybarrow Dodd

Full Resolution (126MP)

Ullswater

Full Resolution(57MP)

Ben Nevis

Full Resolution (167MP)

Blackwater Reservoir

Full Resolution (96MP)

Or if you’re wondering how this technique would apply to a 12MP capture series (where I just took regular old photos). Here is one from the top of Loughrigg, where I forgot to turn on the 48MP mode. It is still, I think, better than what a Camera.app pano would look like but doesn’t quite have the sharpness.

Full Resolution (50MP)
David Smith




Independent as in Freedom, not Independent as in Alone

I’m delighted to announce that I have brought on Stephen Hackett to help me with my apps. He will be working on customer communication, marketing and generally creating space for me to focus on the programming oriented parts of my work.

Something that’s been rattling around my mind recently is the phrase “Independent as in Freedom, not Independent as in Alone”. For so long I think I have been conflating those two ideas in my head. Which has not been serving me well.

I am extraordinarily proud of being an “indie”, it is a meaningful part of my professional identity. As such I held on too long to a sense of needing to do it all myself. But I’ve grown in this regard and I am extremely excited about what Stephen and I will be able to accomplish together.

My personal definition of being an “indie” has grown and been improved upon. It isn’t about being alone, it’s about the freedom to choose your own path and then walk it in the manner aligned to your own values. That part of the indie life I don’t expect to ever give up, but I can walk that path with others and expect the journey to be all the richer as a result.

David Smith




Pedometer++ 5.3: The Ultimate Hiking Companion

Earlier this year I released a major update to Pedometer++ which included a complete visual redesign of the app and brought workout tracking to the iPhone. This update represented a movement towards making Pedometer++ your best companion for outdoor walking adventures.

Today I’m releasing version 5.3 which completes this movement by rounding out some of the missing features from the v5 update. Specifically this update adds Route Planning and Offline Map Management.

I am a very avid hiker. It is my favorite activity and simply put it is my happy place. Because I’ve spent so many hours hiking I’ve developed a number of very strong opinions about what features are important for hiking and how best to build them. These features are built from the perspective of how I plan and track my hikes, developed with the benefit of countless adventures.

Route Planning

While you can continue to import GPX files from external sources into Pedometer++, I wanted to also create a method for planning the routes directly in Pedometer++.

I tend to use GPX files when I am new to an area and want to benefit from other people’s experience. There are numerous hiking trail resources online which publish the best routes in an area and are a valuable way to get familiar with a location.

After walking in an area for a while, however, I find that I typically want to start striking out on my own routes and find new places and hidden gems. There are a number of ways to build a route planner but my favorite method is to boil down a hike into a few key waypoints/viewpoints and then backwards plan a route between them. This is exactly how I’ve built the route planner for Pedometer++.

You simply tap on the locations you want to visit and it will use the Mapbox Directions API to find the shortest route between them. This typically serves as a great starting point for a route. While not necessarily the ‘best’ route, these automatic routes can make it super quick to plan a hike. I’ll often then tweak the automatic route to my tastes based on terrain, access or trail popularity.

Because this planning system is so straightforward and automated it was even possible to add it into the Apple Watch app as well.

I’ve found this super helpful for when I’m actually out on a hike and want to quickly consider an alternative path. Rather than pulling out my iPhone and looking there, I can just tap “Plan a Route” on my wrist, tap a couple of waypoints and very quickly get a distance/route estimate for the possible detour. Then if I like the option I can simply save the new route and use it for the rest of hike, or until the detour is complete.

Offline Maps

Another important feature being added in this update is the ability to more widely download maps for offline use. Rather than just being able to download the map tiles for a particular route you can now download maps for a wide area before you head off on a hike.

This works great for situations where you may be entering an area with limited connectivity. The maps on your iPhone are automatically available on your Apple Watch (as long as your iPhone is within range of your watch).

Ordnance Survey Maps

In the United Kingdom Ordnance Survey maps are the gold standard for outdoor navigation. They provide rich detail for walking routes and rights-of-way. Thankfully they are offered as an API which other apps can make use of and so I’ve been able to include them in this update.

This is also available on your wrist during workouts on your Apple Watch.

Visual Refresh

Lastly I’ve also done a lot of work to improve the visual design of both the iPhone and Apple Watch apps. The old design was feeling a bit “heavy” and cumbersome. I wanted to bring forward a design which felt more modern, clean and intuitive.

On the Apple Watch side of things I had done a partial update this September to bring the app more in line with the watchOS 10 design language. This update completes that work and fully embraces the new layering and visual aesthetic of watchOS.

I hope you enjoy this update, which is available on the App Store now.

David Smith




Matching the Modular Ultra watch face with SF Font Alternatives

Design Notes Diary

The introduction of the second generation Apple Watch brought with it the addition of a new watch face called Modular Ultra. I’m always on the look out for a new digital watch face and this one is very nice, and particularly flexible in terms of layout and look. It also includes a delightful new font showing the time with a variety of alternate heights and widths.

I had the idea for a semi-minimalist layout showing the five things I’m most regularly wanting to see on the face:

  • Time
  • Day
  • Date
  • Temperature
  • Conditions

I could then put these into the four corners of the watch face and end up with a nice clean look. Here was the initial result:

Not too bad but the font shown in the complications (using Watchsmith), just didn’t fit at all with the new Ultra face showing the time.

I’ve heard that this new font used on the Modular Ultra is referred to as Zenith within Apple so I’ll use that name in this article for clarity. I have no idea if that is actually true but calling it the “New time font used in the Modular Ultra face” would be rather cumbersome, so Zenith will do…both for clarity, and also because that is just a super awesome name.

Zenith has a number of font attributes very similar to San Francisco, but looking at the font it also has a number of tweaks and adjustments which make it not match well when shown on the same watch face. I kinda wish that watchOS would have automatically rendered complications in a matching font (like they do on the other Ultra face, Wayfinder), but they don’t as far as I can tell.

So I set out to see if I could adjust regular San Francisco to match Zenith better. The first step was to create a little test app to be able to quickly compare the font rendering options.

The most obvious problem is that the numerals “6” and “9” have curly tails in regular San Francisco, rather than the straightened ones in Zenith. This I can fix by adjusting one of the optional features in San Francisco. Specifically the rather awkwardly named kStylisticAltOneOnSelector.

This leads to this rendering for the “6” and “9”.

Great, but now let’s look at the “4” numeral. Which is closed on San Francisco, but the top of the “4” in Zenith is open. This can be adjusted by kStylisticAltTwoOnSelector.

So now we have a numeral which look s like this:

Getting close but the width and weight of the font aren’t right. But thankfully variable width rendering was recently added to San Francisco so we can now adjust that too.

Leading to a look which is like this:

To my eye that is very, very close. I’m sure there are more typographically adept folks who could tweak or adjust things to make it even more of a match, but this is good enough for my ability.

The last step was in doing a full numeral test to make sure I wasn’t missing something in one of the other numerals.

That looks great to me. So I then took the font I’ve now made and loaded it up into a private build of Watchsmith, and boom…this is the result:

I love the way this face looks. It feels modern but in a way which is harmonious and friendly to me. And the best part, as opposed to some of my previous explorations into building custom watch faces, this is 100% built using the standard components so runs on my wrist without any workarounds or hacks. Delightful.

David Smith




Calculating a Smooth Clock Hands Animation

Design Notes Diary

Let’s start out this week with a little brain teaser type problem.

How would you calculate the rotation angle for the minute and hour hand of a clock?

Specifically this came to mind for me because of a feature in Widgetsmith where you can specify an analog clock as one of your widgets which looks like this.

I’d encourage you to pause for a moment and actually think how you’d approach this because the result I ended up with was way more complex than I would have initially guessed and it was a good learning exercise to reason through.

The version of this feature which shipped with iOS 17 used the rotation angle calculation I had used since Widgetsmith was first created which is based on a simple method dividing a full rotation of the clock hands by the current hour/minute.

This worked fine in the old version of WidgetKit which only showed one widget at a time, but starting in iOS 17 each progressive widget refresh is now animated between the previous and next value. So now at the end of every hour you get this:

Not great, because I’m only calculating each rotation based on a single rotation around the clock face it jumps from 360° back to 0°.

OK I thought let’s adjust the minute so that it takes into account the hour of the day as well and successfully add in an additional 360° rotation at the start of each hour.

That solves the minute hand jumping around during the day, but now at midnight we have this:

Now at midnight we get a massive backwards rotation because we are again reverting to 0° at the start of each day.

So my next thought is that we need to instead try and make the rotation increase continuously (monotonically for the mathematically inclined). That way the rotation will just keep rolling around and around over time.

This was my first attempt at this type of approach where I pick an arbitrary anchor date and then calculate the number of seconds since that date and then just keep rotating based on the number of hours/minutes it has been since then.

This gets around the midnight reset problem. Though it does mean that I am now providing rotations way outside of the typical 360° range so I wanted to then check if this would eventually overflow and cause issues with the renderer. But trying it with a date far into the future seems to work just fine.

But now the next problem I face is a bit more subtle and relates to the spectre which haunts all programming work which relates to time, daylight savings. Because this approach starts its rotation at midnight on New Year’s Day and then increases linearly from there it will fall apart when the clocks change.

I’m not accounting for the fact that there can be instances where the rotation angle isn’t actually evenly increasing between each date. It needs to either jump forward or backwards when the daylight savings points are met.

My first thought for how to solve this problem would be to determine the starting angle of each day and then use that as the reference point to adjust then based on the previous hour/minute method. This way I’m determining the daily rotation based on the actual hour/minute value (2pm, 4:12am, …) and not just the time since the reference.

This approach however includes a subtle bug. Can you spot it? The issue comes from the fact that the start of each day isn’t actually a multiple of 24 hours from the start of the year…because in March when the clocks change we have a non 24 hour day. 🤦🏻‍♂️

So taking this approach I would get funny rendering bugs after March.

But I think I was on the right path by referencing the start of each day as my baseline for then adjusting a daily rotation. But instead of basing it on the number of seconds from the start of the year I need to instead determine the number of whole days and then multiply that out to get how many full daily rotations have occurred.

This is what I ended up with (code here):

Here I use the number of full rotations of each of the hands per day as the basis for my calculations (2 for the hour hand and 24 for the minute hand).

Then determine the number of whole days have past since my anchor date, and multiply this by the revolutions per day.

Now I have the correct starting point from which I can then determine how far to rotate based on the nominal hour and minute values in the current timezone. Then I’m adding these two values together to get the final rotation.

As far as I can tell this works perfectly. I’m still doing a bit more testing to be sure but here is for example what it does at the two daylight savings points:

The animation actually now involves the correct adjustment being made (either jumping forward or falling behind).

Code like this is always an interesting challenge to get right. Personally I find it very difficult to think through all the possibilities and ensure that I’m accounting for all the correct factors.

I hope this approach is right (if you see a bug in my logic please do let me know!), but either way I’ve learned a bunch for the process of thinking it through which was a great way to start out my week.

David Smith