Santiago Villa Cardenas Posted July 19, 2022 Share Posted July 19, 2022 I am currently working on a tool to visualize data but it will be used to analyze many different files, so I need to create an import function that will allow for selecting files and loads it into Spotfire. Link to comment Share on other sites More sharing options...
Solution Gaia Paolini Posted July 25, 2022 Solution Share Posted July 25, 2022 Assuming that you have already a data table in Spotfire containing data that has the same structure as what you want to load, you could use something like this, where "table" is an input parameter of type Data Table (your target table, pre-existing). If your files are all of different structures instead, you will need to also create the data table on the fly. https://community.spotfire.com/s/article/IronPython-script-to-load-an-In-memory-csv-file-in-TIBCO-Spotfire# Copyright © 2022. TIBCO Software Inc. Licensed under TIBCO BSD-style license.# Replace tables from filesimport clrimport Spotfire.Dxp.Data.DataTablefrom Spotfire.Dxp.Data import *clr.AddReference("System.Windows.Forms")from System.Windows.Forms import OpenFileDialogfrom Spotfire.Dxp.Data import *myDataManager = Document.Datad1 = OpenFileDialog()d1.Multiselect=True #lets you select more than one filed1.InitialDirectory='C:\MyData' #the folder containing your source data filesd1.ShowDialog()files=d1.FileNames#lazy workaround: replace data with first file then add the other selected filesds=myDataManager.CreateFileDataSource(files[0])table.ReplaceData(ds)for ff in files[1:]: ds=myDataManager.CreateFileDataSource(ff) settings = AddRowsSettings(table,ds) table.AddRows(ds,settings) Link to comment Share on other sites More sharing options...
Gaia Paolini Posted July 25, 2022 Share Posted July 25, 2022 looks like indentation is lost Link to comment Share on other sites More sharing options...
Santiago Villa Cardenas Posted July 26, 2022 Author Share Posted July 26, 2022 Thank you! Any ideas on how to create a different Table for each selected file? (With this code each file is added together in one single table) Link to comment Share on other sites More sharing options...
Gaia Paolini Posted July 26, 2022 Share Posted July 26, 2022 can you elaborate on what the desired outcome is? In order to create a new table, you will need to know or infer the data structure of each file. Are these files all different or is there a common data structure? Link to comment Share on other sites More sharing options...
Santiago Villa Cardenas Posted July 26, 2022 Author Share Posted July 26, 2022 So currently what it is doing is adding the files together row-wise. Basically all the files that will be used have the same structure. This is what it does now if I select these two files for loading:but what I would want is to have them separated and each one being its own table. Link to comment Share on other sites More sharing options...
Gaia Paolini Posted July 26, 2022 Share Posted July 26, 2022 Actually it is easier than I thought, you only need the data types if they are wrongly inferred. So this should work.Beware of the indentation, I cannot find a way to paste without losing it.# Copyright © 2022. TIBCO Software Inc. Licensed under TIBCO BSD-style license.# Add or replace tables from filesimport clrimport Spotfire.Dxp.Data.DataTablefrom Spotfire.Dxp.Data import *clr.AddReference("System.Windows.Forms")from System.Windows.Forms import OpenFileDialogfrom Spotfire.Dxp.Data import *myDataManager = Document.Datad1 = OpenFileDialog()d1.Multiselect=True #lets you select more than one filed1.InitialDirectory='C:\MyData' d1.ShowDialog()files=d1.FileNamesfor ff in files: tableName = ff.split('\')[-1] #the file name without path ds=myDataManager.CreateFileDataSource(ff) try: newTable = Document.Data.Tables.Add(tableName, ds) except: newTable = Document.Data.Tables[tableName] newTable.ReplaceData(ds) Link to comment Share on other sites More sharing options...
Santiago Villa Cardenas Posted July 26, 2022 Author Share Posted July 26, 2022 Works perfectly, Thank you Gaia!! Link to comment Share on other sites More sharing options...
Sundaram Sridharan Posted November 15, 2022 Share Posted November 15, 2022 I love the original answer that could apply to my situation - i.e. concatenate all input files of same structure into one table - I have a couple of twists that I would like see how to address though:I have a fixed number of header lines I need to remove from each fileI also need to add the file name as a new column (for multi-select file option)is there a way to tweak this "createfiledatasource" method to do the above two things? Link to comment Share on other sites More sharing options...
Gaia Paolini Posted November 16, 2022 Share Posted November 16, 2022 Hello Sundaram. To keep the issues separated, I recommend that you create a new question for this. I will be monitoring the community for your new question. You could link to the original question for additional background. Link to comment Share on other sites More sharing options...
Ming Qian Posted November 17, 2022 Share Posted November 17, 2022 hello allwho can share a document to explain ironpython code, such as "myVis = myVis.As[Visualization]()"?we do not know why the author write it like this, so need to know the reason!Thanks all! Link to comment Share on other sites More sharing options...
Gaia Paolini Posted November 17, 2022 Share Posted November 17, 2022 please can you write this as a separate new question, as it is not related to the previous question. Link to comment Share on other sites More sharing options...
Heleen Snelting Posted November 17, 2022 Share Posted November 17, 2022 Hi @Ming Qian can I kindly ask you to post new questions as a new question with its own title rather than adding to an existing thread? That way our IronPython experts can more easily recognise the question and will respond to you.Thanks,Heleen Link to comment Share on other sites More sharing options...
Ming Qian Posted November 17, 2022 Share Posted November 17, 2022 Hi, @Heleen Snelting you can put it a new question to post, we need more help to better use ironpython in Spotfire like python.Thank you so much! Link to comment Share on other sites More sharing options...
Santiago Villa Cardenas Posted November 21, 2022 Author Share Posted November 21, 2022 Hi Gaia,Getting back to this question since I have run into some issues overtime. Mainly regarding data type handling on load. Most of my files have identical structure but some columns (named equally) have different data types which creates column duplication, for example Column, Column (1), where the first is one specific data type and the second one is a different data type. My objective is to include something in the script above that can help handling these data type issues. Any ideas?Thanks a lot@Gaia Paolini Link to comment Share on other sites More sharing options...
Gaia Paolini Posted November 22, 2022 Share Posted November 22, 2022 which script are you using? By the error it looks like you are trying to merge the data tables into one, but my latest script keeps them separate. Please describe your current use case in detail. Link to comment Share on other sites More sharing options...
Emma Wilson Posted November 9, 2023 Share Posted November 9, 2023 Hello,To import data into Spotfire using IronPython, follow these simple steps:Get IronPython:Make sure you have IronPython installed on your system.Open Spotfire: Launch Spotfire, and open the project where you want to import the data.You can also check this: https://community.spotfire.com/s/article/IronPython-script-to-load-an-In-memory-csv-file-in-TIBCO-SpotfireWrite a Script: In Spotfire, go to the #Tools menu, choose "IronPython Script Editor," and write a script to import the data. Here's a basic example:from Spotfire.Dxp.Application import Documentfrom Spotfire.Dxp.Data import DataTable # Create a new data tablenew_table = DataTable() # Let users select files (you can use a simple file dialog) # Read and load data into the data table# For CSV files, using pandas for simplicityimport pandas as pdfile_path = "path/to/selected/file.csv"data = pd.read_csv(file_path)new_table.ImportRows(data) # Add the data table to the Spotfire documentDocument.Data.Tables.Add(new_table)Run the Script: Execute the script in the IronPython Script Editor.Check Spotfire: Go back to Spotfire, and you should see the data imported into a new table.Remember to adjust the script based on the types of files you're working with and customize it according to your needs. This basic script creates a new data table, lets users select files, loads the data, and adds it to your #Spotfire project.I hope this will help you.Regards@Emma Wilsonsalesforce developer Link to comment Share on other sites More sharing options...
Mach Brown Posted December 28, 2023 Share Posted December 28, 2023 Hello,According to me , To use IronPython for importing data into Spotfire:Import required modules.Prompt users to select files using FileDialog.Create a new data table in Spotfire.Define columns in the data table.Read data from selected files and populate the table.Update the Spotfire visualization to display the imported data. Link to comment Share on other sites More sharing options...
Recommended Posts
Create an account or sign in to comment
You need to be a member in order to leave a comment
Create an account
Sign up for a new account in our community. It's easy!
Register a new accountSign in
Already have an account? Sign in here.
Sign In Now