Can I Park Here (Old) - Dale's Notes

## Description *Can I Park Here* is a mobile app designed to help people determine if they can park at a specific spot. It uses the camera to read parking signs and tells users if they are allowed to park there, what time they must leave, and more. It uses the device's camera to send an image to the server, where the server will then - save that image for later review - detect boundaries for street signs - create objects for each sign - parse text within each sign - comprehend the provided sign data all to determine if a user is allowed to park there. ## Designs ### Design 1 - [[LLaVA]] [[LLaVA]] is supposed to be able to accept images and text for question answering and conversation. The idea is to setup [[LLaVA]] as an assistant/agent and see what the results are. ## [[ChatGPT]] Prompts ### How Would I Ship a [[PyTorch]] Model into Production >Shipping a [[PyTorch]] model to production involves several steps >1. **Model Training**: Train your [[PyTorch]] model using the desired dataset >2. **Model Serialization**: Save the trained model's state dictionary or the entire model using `torch.save()`. This allows you to load the model in the production environment. >```python >torch.save(model.state_dict(), 'model_weights.pth') >``` >3. **Inference Code**: Write code for model inference in your production environment. This typically involves loading the saved model and using it to make predictions on new data >```python >model = YourModelClass() >model.load_state_dict(torch.load('model_weights.pth')) >model.eval() >``` >4. **Input Preprocessing**: Ensure that the input data in the production environment is preprocessed in the same way it was during training. This may include normalization, resizing, or any other necessary transformations >5. **Dependencies**: Install the required dependencies in your production environment. It's recommended to use a virtual environment to manage dependencies >6. **Scalability**: Consider how your model with handle production-level traffic. Tools like [[TorchServe]] or deploying your model with cloud services ([[AWS SageMaker]], [[Google AI Platform]]) can help with scalability >7. **Monitoring and Logging**: Implement logging and monitoring to keep track of your model's performance in the production environment. This helps in debugging and improving the model over time. >8. **Security**: Secure your model by controlling access, encrypting sensitive information, and following best practices for securing machine learning models in a production environment. >9. **Testing**: Thoroughly test your model in the production environment to identify and address any issues that may arise. Consider A/B testing to compare the new model's performance against the existing one. >10. **Documentation**: Document your deployment process, including dependencies, preprocessing steps, and any other relevant information. This ensures that others can understand and replicate the deployment. ## Work Log - [[2023-11-27]] - Updated iPhone to iOS 17 and that requires me to update XCode to 15 beta - Still having trouble with this phantom error when trying to use the camera in iOS emulation - I'm actually following the [Ionic Camera](https://ionicframework.com/docs/vue/your-first-app/adding-mobile) documentation to see if that helps with anything - In order to use XCode 15 beta, I have to upgrade MacOS to Sonoma 14.x - [[2023-11-28]] - Try using [[ChatGPT]] - [x] Take Silvie's picture - [x] Send it to [[ChatGPT]] in a [[zero-shot prompt]] - I created an account with OpenAI Developers and got access to platform.openai - Decided not to go through - [[2023-11-29]] - [[ChatGPT Alternatives]] - [[2023-12-09]] - Training & refinement - Business model - Marketing model - [[2023-12-27]] - [x] Setup [[Jupyter Notebook]] again - Run `jupyter notebook` in `~/projects/can-i-park-here` - Notebooks are in `./notebooks` - [x] Load [[blip-vqa-capfilt-large]] model - This isn't that promising because the responses are pretty bad. There is probably too much noise in the image and I'll probably need to redefine some steps - [x] Try uploading a file to the model ## User Experience 1. User opens app 1. App opens to phone and geolocation settings 2. Phone app provides some guidance for boundary detection (tap on the screen to find squares) 2. User takes picture of sign with metadata 1. Geolocation 2. Signs unable to be parsed 3. Phone sends picture to server 4. Server 1. Saves photo 2. Parses photo 3. Sends back data to phone 5. Phone displays "Green Yes" or "Red No" for a few seconds 6. Phone also displays follow-up options such as 1. Set alarm for when driver has to leave 2. Set alarm for when parking spot is available 3. Visualize available times differently ### Edge Cases - Signs that cannot be parsed - Signs that have images - When the user tries to be deceitful ## Tech Stack and Development ### Front-end On the front-end I'm using [[Quasar]], which is a JS/TS framework that deploys apps in a variety of platforms. Specifically I'm using the [Quick Start - Quasar](https://quasar.dev/start/quick-start) to learn first with the Bun/Vite runtime underneath. However [[bun.sh]] still has issues with Darwin 64, so I'm switching back to [[yarn]]. Quasar only runs on Node 18 and lower. Run `quasar dev` in the folder `~/projects/quasar-project`. This project uses [[VueJS|Vue]] so I'll have to brush up on learning [[Vue]] with the [[Composition API]]. I want to enable [[Capacitor]] in this project so that I have access to the Camera API. I'll also need access to the GPS API too. When adding [[Capacitor]] to the project, I got asked about a `Capacitor app id`. Building and running the app on my phone shows a pop-up: > Untrusted Developer > Your device management settings do not allow using apps from developer "Apple Development: [email protected] (H8W7C627ZQZ) on this iPhone. You can allow using these apps in Settings" I got the app to run with help from [Apple Developer Forum](https://developer.apple.com/forums/thread/660288), specifically `Settings > General > Device Management > select the profile to trust`. [Adding Capacitor APIs to the project](https://quasar.dev/quasar-cli-webpack/developing-capacitor-apps/capacitor-api#using-a-capacitor-api) - [x] Geolocation - [x] I need to add the `@capacitor/geolocation` dependency - [x] Add routing from `IndexPage` to `GpsPage` - Update some [Configuring Info.plist](https://capacitorjs.com/docs/ios/configuration#configuring-infoplist) for location privacy - Camera Sometimes when rerunning the CLI after installing a dependency, Quasar will as for an IP to use. I use `192.168.1.66`. ### Back End Ideas - Option 1: ChatGPT and prompt engineering - Option 2: Free Alternative to ChatGPT - Option 3: Free Alternative to ChatGPT - Option 4: Free Alternative to ChatGPT ### Model Layer #### Training Aspects - Postgres to describe the relationships between - Image - id - image - coordinates - Model - id - name - type - Prompt - id - type - question with blanks - Prompt Inputs - id - array of inputs - Results - id - imageId - modelId - promptId - promptInputsId - response - actual #### Fine-tuning Aspects Basically I'll need to review the performance of all of the models to see if they're accurate. If they're not right, then I need to be able to update the model in some way via fine-tuning. ## Business Model ### Designing a Business Model I think I'd need to keep track of [[Business Expenses]] in order to figure out a pricing structure. If I can't cover my costs, then there isn't really a point to ### Options #### Free Trial This model allows people to try the app for free before needing to pay to use the app. There are varying paid models to use afterward, since this is a SaaS. ## Marketing Model There's almost no "build it and they will come" model here. Everyone's attention is starved because everyone is being inundated with information from all directions. In order to get people to sign on, I need to evoke a strong enough emotion in them to make them want to install the app, sign up, etc. I can get sign-up metrics from the various app stores. Additionally I can get usage metrics from the site also. Maybe [[Datadog]] has an open-source tier I can deploy to the cloud. I also asked [[ChatGPT]] what I should do for marketing and it came up with the following in [[Marketing Prompt]] ### Social Media I will likely need some kind of continuous social media presence. I could use - human labor - automated posting - automated content generation ## References - [[Tech Ideas MOC]]